NixOS Package

[gale_username]

Hello! I've been trying to make a package for Drag[en]gine on NixOS. I'm currently using 5 patches for this but still run into a non-obvious to me compilation error. First I'll go through the patches I had to make.

This one lets me add to CPPPATH and LIBPATH:

Code: Select all

diff --git a/SConstruct b/SConstruct
index d4080e9c3..4f67cfd11 100644
--- a/SConstruct
+++ b/SConstruct
@@ -154,6 +154,9 @@ if not (parent_env['OSPosix'] or parent_env['OSWindows'] or parent_env['OSBeOS']
 #params = Variables(['parameters.cache', 'custom.py'])
 params = Variables(['custom.py'])
 
+params.Add(PathVariable('append_cpppath', 'Additional paths to search for include files', None))
+params.Add(PathVariable('append_libpath', 'Additional paths to search for libraries', None))
+
 params.Add(EnumVariable('platform_android', 'Build for Android platform', 'no', ['no', 'armv7', 'armv8', 'x86']))
 params.Add(BoolVariable('with_tests', 'Build engine tests', False))
 params.Add(BoolVariable('with_debug', 'Build with debug symbols for GDB usage', False))
@@ -535,6 +538,11 @@ else:
 params.Update(parent_env)
 #print(parent_env.Dump())
 
+if parent_env['append_cpppath']:
+	parent_env.Append(CPPPATH = Split(parent_env['append_cpppath']))
+if parent_env['append_libpath']:
+	parent_env.Append(LIBPATH = Split(parent_env['append_libpath']))
+
 # determine sanitize flags to use
 parent_env.Replace(SANITIZE_FLAGS = [])
 parent_env.Replace(SANITIZE_LINK_FLAGS = [])

This one allows the use of system openal. Are you able to clarify what the "annoying limitations" are that disable this by default?

Code: Select all

diff --git a/extern/openal/SConscript b/extern/openal/SConscript
index fd6727c26..355fd7ee7 100644
--- a/extern/openal/SConscript
+++ b/extern/openal/SConscript
@@ -34,7 +34,7 @@ else:
        staticFileName = envLibrary.subst('lib/${LIBPREFIX}openal${LIBSUFFIX}')
 
 # check if system openal is present
-hasSystemOpenAL = False
+hasSystemOpenAL = True 
 """ force false due to annoying limitations
 conf = envLibrary.Configure()
 if envLibrary['with_system_openal'] != TernaryVariableNo:

Next one allows for using the system openvr:

Code: Select all

diff --git a/extern/openvr/SConscript b/extern/openvr/SConscript
index f507cad0f..e6bc74e5c 100644
--- a/extern/openvr/SConscript
+++ b/extern/openvr/SConscript
@@ -36,6 +36,16 @@ else:
 	libLinkName = 'openvr_api'
 	libpath = [envLibrary.Dir('lib')]
 
+parent_targets['lib_openvr'] = {
+	'name' : 'Internal OpenVR library',
+	'cpppath' : [],
+	'cppflags' : [],
+	'libpath' : [],
+	'libs' : ['openvr_api'],
+	'runtimelibs' : [],
+	'depends' : [] }
+Return()
+
 def openvrBuild(target, source, env):
 	buildDir = target[0].get_dir().up()
 	

Same for openxr:

Code: Select all

diff --git a/extern/openxr/SConscript b/extern/openxr/SConscript
index fa2acc0ad..f3b7fa51b 100644
--- a/extern/openxr/SConscript
+++ b/extern/openxr/SConscript
@@ -36,6 +36,16 @@ else:
 	libLinkName = 'libopenxr_loader'
 	libpath = [envLibrary.Dir('lib')]
 
+parent_targets['lib_openxr'] = {
+	'name' : 'Internal openxr librar',
+	'cpppath' : [],
+	'cppflags' : [],
+	'libpath' : [],
+	'libs' : [libLinkName],
+	'runtimelibs' : [],
+	'depends' : [] }
+Return()
+
 def openxrBuild(target, source, env):
 	buildDir = target[0].get_dir().up()
 	

Finally, there's a patch to fix the libLinkName for sndio:

Code: Select all

diff --git a/extern/sndio/SConscript b/extern/sndio/SConscript
index 61abce950..982c508f9 100644
--- a/extern/sndio/SConscript
+++ b/extern/sndio/SConscript
@@ -16,7 +16,7 @@ buildStaticLinking = True
 srcdir = 'sndio-1.6.0'
 
 libLinkVersion = '0'
-libLinkName = 'ogg'
+libLinkName = 'sndio'
 
 if envLibrary[ 'OSWindows' ]:
 	libFileName = 'bin/lib{}-{}.dll'.format( libLinkName, libLinkVersion )

The compilation error I get is:

Code: Select all

Compiling src/launcher/gui/src/gui/games/properties/deglDialogGameProperties.cpp
src/launcher/gui/src/gui/games/properties/deglDialogGameProperties.cpp: In static member function 'static FX::FXint deglDialogGameProperties::fSortCache(const FX::FXIconItem*, const FX::FXIconItem*)':
src/launcher/gui/src/gui/games/properties/deglDialogGameProperties.cpp:1270:32: error: 'compare' was not declared in this scope
 1270 |                         return compare( cache1.name, cache2.name );
      |                                ^~~~~~~
scons: *** [src/launcher/gui/build/src/gui/games/properties/deglDialogGameProperties.o] Error 1
scons: building terminated because of errors.

[dragonlord]

Hi gale_username. I'll try to answer the question as good as possible. Can you also state which version of Drag[en]gine you try to package?

Patch 1:
It is possible to do it that way but this will not affect external build software. This is because external builds are not affected by SCons LIBPATH nor LINKPATH. Instead they use CXXFLAGS, CPPFLAGS and LDFLAGS. You should instead of this patch define environment variables like 'CPPFLAGS="-Iincpath1 -lincpath2"' and 'LDFLAGS="-Llibpath1 -Llibpath2"'. These will be also picked up by external build software. The relevant part in the SConstruct can be found here.

Patch 2:
The problem with the upstream OpenAL is a bug in the effect handling causing and exception shutting down the game engine. This is the patch applied to fix the problem. If you build using upstream OpenAL the game engine can run for some time and then suddenly fail in the OpenAL somewhere. Now NixOS seems to use OpenAL 1.23.1 . Maybe this problem does not exist there. I should re-allow using system OpenAL with potentially or randomly shut down the audio module for no reason.

Patch 3 and 4:
I would personally not use system OpenVR and OpenXR system wide packages (aka shared libraries). The libraries are not designed for this. But I can also add support for using system packages for those wanting to experiment with such a setup.

Patch 5:
You can drop this patch. Sndio is deprecated and not used anywhere in the game engine. I should remove this old package.

Now concerning the compile error this is caused by a breaking change in FOX toolkit around some version. The game engine uses 1.7.67 and NixOS seems to be using 1.7.81 . Looks like the compare() function has been killed now for good. You can fix this problem by altering this line removing the OLD_STRING_COMPARE_NS compile definition. This will cause the newer "FXString::compare()" to be used. I guess it would be a good idea to add a build time test to figure out if this compile definition is set or not.

I'll add the changes tomorrow to improve the situation.

[dragonlord]

Concerning OpenAL I checked the sources. Current head has the same fix applied so building against system OpenAL should be safe with version 1.23.1 and newer.

[gale_username]

Thanks for the help!

Can you also state which version of Drag[en]gine you try to package?

Always the latest released version, so 1.23 at the moment.

You can drop this patch. Sndio is deprecated and not used anywhere in the game engine. I should remove this old package.

I can switch to disabling it, but I think that would still need a patch.

Now concerning the compile error this is caused by a breaking change in FOX toolkit around some version.

There now seem to be a few other seemingly Fox related errors:

Code: Select all

Compiling src/deigde/deigde/shared/src/engine/igdeEngineController.cpp
In file included from src/deigde/deigde/shared/src/engine/../gui/native/fox/foxincludenative.h:62,
                 from src/deigde/deigde/shared/src/engine/../gui/native/toolkit.h:32,
                 from src/deigde/deigde/shared/src/engine/igdeEngineController.cpp:34:
src/deigde/deigde/shared/src/engine/../gui/native/fox/igdeNativeFoxWidget.h:70:16: error: 'Window' does not name a type
   70 |         static Window NativeWidgetID( const igdeWidget &widget );
      |                ^~~~~~
src/deigde/deigde/shared/src/engine/../gui/native/fox/igdeNativeFoxWidget.h:78:16: error: 'Window' does not name a type
   78 |         static Window NativeWidgetParentID( const igdeWidget &widget );
      |                ^~~~~~
   78 |         static Window NativeWidgetParentID( const igdeWidget &widget );
      |                ^~~~~~
src/deigde/deigde/shared/src/engine/../gui/native/fox/igdeNativeFoxWidget.h:89:16: error: 'Display' does not name a type
   89 |         static Display *GetDisplayConnection();
      |                ^~~~~~~
src/deigde/deigde/shared/src/engine/igdeEngineController.cpp: In member function 'deRenderWindow* igdeEngineController::CreateRenderWindow(igdeWidget&)':
src/deigde/deigde/shared/src/engine/igdeEngineController.cpp:449:51: error: 'class deRenderWindowManager' has no member named 'CreateRenderWindowInside'; did you mean 'CreateRenderWindow'?
  449 |         return pEngine->GetRenderWindowManager()->CreateRenderWindowInside(
      |                                                   ^~~~~~~~~~~~~~~~~~~~~~~~
      |                                                   CreateRenderWindow
src/deigde/deigde/shared/src/engine/igdeEngineController.cpp:450:35: error: 'NativeWidgetParentID' is not a member of 'igdeNativeWidget' {aka 'igdeNativeFoxWidget'}
  450 |                 igdeNativeWidget::NativeWidgetParentID( hostWindow ) );
      |                                   ^~~~~~~~~~~~~~~~~~~~
src/deigde/deigde/shared/src/engine/igdeEngineController.cpp: In member function 'void igdeEngineController::pCreateMainRenderWindow()':
src/deigde/deigde/shared/src/engine/igdeEngineController.cpp:607:72: error: 'class deRenderWindowManager' has no member named 'CreateRenderWindowInside'; did you mean 'CreateRenderWindow'?
  607 |                 pMainRenderWindow = pEngine->GetRenderWindowManager()->CreateRenderWindowInside(
      |                                                                        ^~~~~~~~~~~~~~~~~~~~~~~~
      |                                                                        CreateRenderWindow
src/deigde/deigde/shared/src/engine/igdeEngineController.cpp:608:43: error: 'NativeWidgetID' is not a member of 'igdeNativeWidget' {aka 'igdeNativeFoxWidget'}
  608 |                         igdeNativeWidget::NativeWidgetID( pMainWindow ) );
      |                                           ^~~~~~~~~~~~~~
scons: *** [src/deigde/deigde/build/shared/src/engine/igdeEngineController.os] Error 1
scons: building terminated because of errors.

[dragonlord]

The sndio should not need a patch nor disabling since no package has it as dependency. Hence SCons will never pull it into the build process automatically.

Concerning the IGDE build this line is caused by missing X11.

[gale_username]

After enabling "build_input_x" and "with_x", I'm having a lot of trouble getting it to recognize the X11 header files. All the system dependencies are passed via "CPPFLAGS" and "LDFLAGS" as a single include folder and single single lib folder. Most checks work fine like the webp one, but both the "X11/Xlib.h" and "X11/extensions/Xrandr.h" checks fail. The only thing that succeeds is "conf.CheckLib('X11')".

Do you have any ideas on what the problem could be? The header files do exist.

[dragonlord]

I don't know. These headers are standard locations. Has NixOS a unique way of handling X include files?

I googled quickly and NixOS seems to be quite peculiar in handling this. Maybe these links help you out?
- https://discourse.nixos.org/t/cant-comp ... nd/12633/6
- https://discourse.nixos.org/t/conflicts ... ders/33498

[gale_username]

Ya. I tried to make things less confusing by wording the question in a way that avoids the many non-traditional behaviors of NixOS.

What finally fixed the problem was including xorgproto, libXrender, libXext, and libXfixes in the generated include and lib folders.

[dragonlord]

So you could get it compiling without changing the Drag[en]gine sources or do I need to include some fixes in a future release?

[gale_username]

I'm able to compile with only patches for what you've already fixed, but there's still a significant problem getting things to run.
Nix uses patchelf to edit paths in shared libraries to their unique Nix locations. This is causing all the file size checks for dragengine modules to fail, because they've been changed since the module manifests were generated.