Repository navigation
Feature request: Compile time argument for models folder in cli #18
Description
I am building a package and installing in a common location to make it easier to use as a global library and am currently patching cli.cpp thus:
src/cli/cli.cpp
--- a/src/cli/cli.cpp 2026-02-06 12:13:30.426359449 +0000
+++ b/src/cli/cli.cpp 2026-02-06 13:56:47.601175698 +0000
@@ -354,8 +354,14 @@
void run_sam(cli_args const& args) {
backend_device backend = backend_init(args);
+
auto [file, weights] = load_model_weights(
+#ifdef MODEL_PATH
+ args, backend, MODEL_PATH "/MobileSAM-F16.gguf", 0, backend.preferred_layout());
+#else
args, backend, "models/MobileSAM-F16.gguf", 0, backend.preferred_layout());
+#endif
+
sam_params params{};
require_inputs(args.inputs, 1, "<image>");
@@ -408,8 +414,14 @@
void run_birefnet(cli_args const& args) {
backend_device backend = backend_init(args);
+
+#ifdef MODEL_PATH
auto [file, weights] = load_model_weights(
- args, backend, "models/BiRefNet-F16.gguf", 0, backend.preferred_layout());
+ args, backend, MODEL_PATH "/BiRefNet-lite-F16.gguf", 0, backend.preferred_layout());
+#else
+ auto [file, weights] = load_model_weights(
+ args, backend, "models/BiRefNet-lite-F16.gguf", 0, backend.preferred_layout());
+#endif
require_inputs(args.inputs, 1, "<image>");
image_data image = image_load(args.inputs[0]);
@@ -452,8 +464,14 @@
void run_depth_anything(cli_args const& args) {
backend_device backend = backend_init(args);
+
+#ifdef MODEL_PATH
+ auto [file, weights] = load_model_weights(
+ args, backend, MODEL_PATH "/DepthAnythingV2-Small-F32.gguf", 0, backend.preferred_layout());
+#else
auto [file, weights] = load_model_weights(
args, backend, "models/DepthAnythingV2-Small-F32.gguf", 0, backend.preferred_layout());
+#endif
require_inputs(args.inputs, 1, "<image>");
image_data image = image_load(args.inputs[0]);
@@ -488,8 +506,15 @@
void run_migan(cli_args const& args) {
backend_device backend = backend_init(args);
+
+#ifdef MODEL_PATH
+ auto [file, weights] = load_model_weights(
+ args, backend, MODEL_PATH "/MIGAN-512-places2-F16.gguf", backend.preferred_layout());
+#else
auto [file, weights] = load_model_weights(
args, backend, "models/MIGAN-512-places2-F16.gguf", backend.preferred_layout());
+#endif
+
migan_params params = migan_detect_params(file);
params.invert_mask = true; // -> inpaint opaque areas
@@ -526,8 +551,15 @@
void run_esrgan(cli_args const& args) {
backend_device backend = backend_init(args);
+
+#ifdef MODEL_PATH
+ auto [file, weights] = load_model_weights(
+ args, backend, MODEL_PATH "/RealESRGAN-x4.gguf", 0, backend.preferred_layout());
+#else
auto [file, weights] = load_model_weights(
args, backend, "models/RealESRGAN-x4.gguf", 0, backend.preferred_layout());
+#endif
+
esrgan_params params = esrgan_detect_params(file);
printf("- scale: %dx\n", params.scale);
printf("- block count: %d\n", params.n_blocks);
src/cli/CMakeLists.txt
--- a/src/cli/CMakeLists.txt 2026-02-05 23:47:15.000000000 +0000
+++ b/src/cli/CMakeLists.txt 2026-02-06 13:16:26.118842969 +0000
@@ -4,4 +4,5 @@
target_compile_definitions(vision-cli PRIVATE ${VISP_ASSERT} ${VISP_DEFINITIONS})
target_compile_options(vision-cli PRIVATE ${VISP_WARNINGS} ${VISP_COMP_OPTIONS})
target_link_options(vision-cli PRIVATE ${VISP_LINK_OPTIONS})
-target_link_libraries(vision-cli PRIVATE ggml visioncpp ${VISP_FMT_LINK})
\ No newline at end of file
+target_link_libraries(vision-cli PRIVATE ggml visioncpp ${VISP_FMT_LINK})
+target_compile_definitions(vision-cli PRIVATE "MODEL_PATH=\"${MODEL_PATH}\"")Note Also changed "BiRefNet-F16.gguf" to "BiRefNet-lite-F16.gguf"
If it would be possible to integrate this, or something similar it would be much appreciated.
Activity
Acly commentedon Feb 9, 2026
Would a search path make more sense? Something like
./models$XDG_DATA_HOME/visioncpp/models$HOME/.local/share/visioncpp/models${SYSTEM_MODEL_PATH}(which might be/usr/local/share/visioncpp/modelsbut a cmake variable to override is fine)By the way how do you intend to handle the ggml libraries when packaging? They currently aren't compatible with upstream ggml (which may be provided as separate package). I try and upstream most changes, but some of it is very vision-specific, and I'm not confident it will be included in near future.
Timh1970 commentedon Feb 9, 2026
Running cli globally would fail to find the default models if they were in a $HOME folder for users logged in with accounts other than that of the installer so I try to keep dependencies of global applications in global locations; that is why I was minded to put the default models in /usr/share/vision.cpp/models when packaging.
I have to admit to paying little attention to where the upstream ggml was coming from so that is a whole new issue, but not insurmountable.
Option 1 (ideal)
Static linking
Option 2
I am thinking install the shared objects in /usr/lib/vision.cpp then when building against vision.cpp it will be a case of
and
I can then bundle a pkg-config file with something like
and then building against vision.cpp should be just a case of using
There is always the possibility of conflicts with the upstream library when linking but paths in PKG_CONFIG_PATH are searched first, so theoretically this version takes precedence.
It was kinda lucky I didn't have libggml alreayd on my test boxes or I would have broken something.
Timh1970 commentedon Feb 10, 2026
I have tested building libvisioncpp while statically linking llama and everything works as expected in principle; the downside is the resultant libvisioncpp.so is 50mb but I still think it is a better option than option 2
Tests show no effect on processing speed so I think static linking is the best solution.
While I appreciate that I will need to track this and manually update I have changed
root CMakeLists.txt:and
visp/CMakeLists.txtI still need to make a package, install it and test but don't see any major challenges. I may maintain a shared version and a static version but set set conficts(llama) in the shared version.
Acly commentedon Feb 10, 2026
Static build for ggml/ggml-base makes sense, I have to try it still, but wouldn't mind making it the default.
For the backends (
GGML_BACKEND_DL) I think it would still make sense to keep dynamic loading. These are not linked at compile time, so they don't need to be on the linker path. Rather ggml tries to dynamically load them at runtime, so it just needs to know where to find them and they can be anywhere.The advantage to keeping the backends dynamic is that people without Vulkan installed (maybe no GPU in the system) can still load and use the library, which would not be possible if the Vulkan backend is statically linked into libvisioncpp. Same for CPU if the minimum instruction set (currently AVX for x86) is not supported.
Definitely install models that ship with the package into the system location. I was thinking of users downloading alternative models eg. there are more precise high-res versions of BiRefNet, but they're more heavy-weight. Having a search path with fallbacks would allow these to co-exist.
Timh1970 commentedon Feb 10, 2026
It will probably be easier to maintain packages with without Vulkan separately; I am still very much experimenting and seeing what works for now.
My rig is not well suited to running on cpu (it's ancient). It does work, but in tests it took on a large file (which I use to try and punish it) it took 9 minuted to process and the cooling fans went for it. Same file with Vulkan took 20 seconds on an old AMD RX 580; on a more sensible file I am getting around 390ms. Interestingly on a nvidia RTX4060 the large file makes it fall over; but that is an nvidia issue as it doesn't manage memory as well (sensible file works okay and runs inference in around 70ms); both tested with BiRefNet-dynamic-F16
Yes, I use BiRefNet dynamic ordinarily but the cli is a great little convenience for quick testing and to give a rough idea of what each model does.
I will have a go at compiling with GGML_BACKEND_DL dynamic and then do some tests to see what happens.
Timh1970 commentedon Feb 10, 2026
Is there any chance you can do a 0.3.1 tag so I can play with my patches? I did a dumb thing and both packaged and made patches without a tag.
Acly commentedon Feb 10, 2026
Not really sure why it's important, but okay: https://github.com/Acly/vision.cpp/tree/v0.3.1
don't think it'll confuse anybody at this point
Timh1970 commentedon Feb 11, 2026
Normally I would just download a release tarball but couldn't do that because sub modules aren't in the tarball so I have to clone it instead. If a line of code changes in any of the files that I have patched then I need to redo the patches. I can't use the normal source() directive in PKGBUILD because
--recursiveis not supported (I always build on Arch first because it has the one of the easiest way to build in a chroot environments, then move on to the more challenging distros later). Instead I need togit clone --recursive --branch v0.3.1 [https://](https://github.com/Acly/vision.cppBuilding against a branch will mean that the patches will always work regardless of how many changes are made.Once everything tests out okay you can decide what (if anything) you wish to incorporate and I can merge the changes with any remaining patches on the next tag.
I messed up because I should have included the
--branch v0.3.0but a few changes have been made since then so when I remembered it was too late (I either re did the patches or got a new tag).I have amended the patches for the CMakeLists as I messed up a bit and it only showed up when I built in a chroot.
src/visp/CMakeLists.txt
root/CMakeLists.txt
The PKGBUILD lets CMake do the installing and then strips out the archives and unnecessary headers afterwards for the static build to avoid conflicts.
Acly commentedon Feb 11, 2026
made some changes: #20
not sure I got it all, but this should allow you to do
and it will install models to /usr/share/visioncpp (as part of cmake --install) and cli will find them there
I've only done a bit of testing on windows so far
NB: this will not install any ESRGAN model, the one downloaded for tests is not really suitable. but now we could define a more generalist upscaler instead that comes with
VISP_INSTALL_MODELSbut notVISP_TESTS.Timh1970 commentedon Feb 11, 2026
Ooooh so close it isn't even worth writing a patch for. Building on the mode-install-path branch gives me
Which is kind of self explanatory.
Line 109 insert 'set(CMAKE_POSITION_INDEPENDENT_CODE ON)` and it compiles.
I will have to do a bit of deeper testing in a couple of days but on the face of it that looks great.
PS. I am afraid that the best I can manage is to fumble around when it comes to CMake but that looks like a much cleaner solution more elegant solution than mine.
I don't have time to do a full test right now but at a glance I think you might need to put a leading
/in VISP_MODEL_INSTALL_DIR as I have a feeling without it it will resolve to /usrshare.Acly commentedon Feb 11, 2026
Ah I was thinking "I will put PIC in a separate commit" and then forgot.
I think for installing it should be fine, but there's an issue when passing it to CLI since VISP_MODEL_INSTALL_DIR is typically a path relative to CMAKE_INSTALL_PREFIX (ie. just "share/visioncpp"). The install command concatenates them and makes "/usr/share/visioncpp", but need to do the same manually for the CLI definition.
Should be fixed now on the branch.
Timh1970 commentedon Feb 18, 2026
So far all seems good (good enough to continue at least). I haven't done exhaustive tests as my workload just got silly, but in principle all is good.
I am thinking now that this works well enough for my use case, but it makes sense now we have gone this far to do it properly if you like.
Next thing would be to make it ABI friendly (and I have no idea how to do that in Cmake, but it can't be that hard) Were it an ordinary makefile then you would:
This gives you a libvisioncpp.so.0.3.0 file and a couple of symlinks to libvisioncpp.so.0 and libvisioncpp.so; which is enough to package it for Debian based distros (ish).
Timh1970 commentedon Feb 28, 2026
It turns out making the ABI compliant library is easier than it looks
While you are at it, if you could create a visioncpp.pc.in file in the project root
and add this to the bottom of the CMakeLists.txt file so that it is installed in the right place, that would be fantastic.
Acly commentedon Mar 2, 2026
I commented out SO version at some point, it was causing issues in some case, have to recheck. But it should be solvable somehow.
Acly commentedon Mar 2, 2026
Yea seems to work fine.
Something else I noticed:
lib/cmake/ggmlgets installed, but should probably not be packagedlibvisioncpp?The library was never meant to be a sealed layer around ggml, but rather something that can be used alongside. Parts of it can be used as building blocks for bigger ggml graphs. I'll try to do some testing with an example project that consumes the package with statically built ggml without linking it. Maybe with little effort the "high-level API" can work like that. And the rest is not so important for now (IMO people who are interested in that would include the lib as part of their source-tree anyway, not link a system package).
Timh1970 commentedon Mar 2, 2026
The issue might have been something to do with position independent code not being enabled previously, but that is resolved already so with any luck everything will work okay. One thing that might have broken is the python bindings; I am no expert but it looks like _lib.py is looking for an unversioned .so
Try something like
Acly commentedon Mar 3, 2026
Why should there be a problem with python? The unversioned .so is still there as symlink.
I think the pkgconfig will not work for now, consuming the lib currently requires ggml, which will conflict.
It can be done in the future, either just making a C API available that does not require ggml, or by aligning with upstream.
Timh1970 commentedon Mar 3, 2026
I am not sure how much you know about it so forgive me if I am sounding a bit patronising, but the Debian and Fedora/Red Hat way is to take a perfectly good sensible package and split it into 3 parts.
Arch style distros don't care, you just bundle the lot together in a single package but Debian and Fedora/Red hat do.
One other little tweak I forgot about was the lib dir varies across distros, but thankfully Cmake automates this if you add this near the top:
Then update the install command to this:
And I think that is it, one way or another that should be sufficient to package it on every distro.
Acly commentedon Mar 3, 2026
This should work out, we just don't do
libvisioncpp-devfor now (and exclude the respective part in arch too).Python I would also exclude - it's usually distributed via PyPi and self-contained.
This is already there (the
VISP_*versions are initialized with those variables).Timh1970 commentedon Mar 3, 2026
My apologies, I missed it somehow.
That should be fine; If I exclude python then doing the -dev package really isn't an issue as it is just split by the .install files inside the debian folder; as it is the first go, it is very much a make it and see what breaks thing at this stage so nothing is set in stone.
I just want to put a template together and see what lintian throws at me and whether it generates all the necessary symbols; Debian has a habit of deciding "Oh you don't need that I will just leave it out!", which inevitably leads to a 2 day fight with conflicting linker flags when it builds perfectly but you can't run anything built against it because of a missing symbol for a random constructor.
Acly commentedon Mar 3, 2026
Okay, merged the changes.
Just keep in mind that to build anything against libvisioncpp the (static) ggml libs and include files are still needed, but they should not be installed into global folders like /usr/lib or /use/include to avoid clash with "official" libggml packages.
Timh1970 commentedon Mar 3, 2026
Got it; I remove the ggml archive and includes as part of the post installation as they are not needed due to the archive being baked in. I did note however that the archive is still exposing its symbols however also noted that it looks like you already prepared to hide them away so I just did.
Alt the top of your CMakelists.txt and
At the bottom and now it is clean so you should be able to install anything alongside it with impunity; these are all that are exposed now and preliminary testing with the built in cli implies it works as expected with no ggml symbols exposed at all.
Acly commentedon Mar 4, 2026
Done, added a
VISP_PACKAGEoption to make this optional because the python tests need internal symbols to work.Setting
VISP_PACKAGE=ONwill also setVISP_INSTALL_MODELS=ONby default.