Add normal pip dependencies

#35
by nikich340 - opened

Being forced to add venv to every local ai is very space-consuming, so I don't want to use uv for that.
Why can't you add normal dependencies file so I can install them globally and run app.py myself?

It also auto-install wrong torch version, which is not compatible with rtx 5070 ti (while I ALREADY have globally installed correct torch version, lol)

So finally I managed to make it work. Seems that uv.lock and pyproject.toml files hardcoded pytorch version and always remove anything installed in venv and replace with pytorch-cu126 (while on Blackwell RTX 5000 cu128 is required!).
So I had to remove these files and manually install all dependencies from pyproject.toml list via pip and then launch usually (python.exe app.py).

Other than that the model is very impressive, cool result!

Being forced to add venv to every local ai is very space-consuming, so I don't want to use uv for that.
Why can't you add normal dependencies file so I can install them globally and run app.py myself?

I get your point about pinning to cuda minor versions for uv (this issue does not apply when using pip directly), in my view they should pin to major unless they depend on functionality which will likely be deprecated/removed in future.

Context: when i say, major/minor/patch i am revering to semantic versioning, which these packages use.

However i don't think the full statements is accurate.

Ive read the pyproject.toml file, and tested locally with system python(v3.13.3) with pip in a venv(i didn't try without a venv as i dont want to pollute my system by installing into global space and break stuff, the same process applies), "uv" is not a hard dependency for dia, one of the main purposes of uv is to be able to install and manage multiple python versions, unless someone goes out of their way to force uv or tightly couple it elseware, i don't see how it has much baring on the install.

Regarding dependencies, a pyproject.toml file is the sucessor to requirements.txt (which is officialy deprecated, and not best practice, you can read the PEP's for more on that), you do not need a virtual environment (venv) to use this tool, its just reccomended because otherwise those dependencies get installed globally, if any other tool using that site packages space relies on a different version of any dependency needed elseware, you can have dependency/version conflicts, this is the main reason why virtual environments are more suitable.

I think Dia has followed best practices here.

If you truly still believe they are wrong, open a PR adding a "requirements.txt" or any other adjustments you want done, to fix it, its easy to criticise, if the raised issue is trivial to fix(like this one is), one might as well fix it themselves and open a PR, even if they don't merge it, any user who would prefer it that way can apply the patch, this is the beauty of free as in freedom/libre software.

Log file, starting Dia app without UV (standard pip in a venv):
https://pastebin.com/raw/J5XtR5DW

Also if you use many "AI" services(which i assume you mean spaces and other software utilising HF libraries/etc), you almost certainly break them (depending on the number of those HF projects you run on the same system). You may find not using a virtual environment counter productive long term in that case.

below i will explain how to generate a requirements.txt from a pyproject.toml, first i would appreciate you read my guidance on this topic (even if you disagree its worth mentioning and i think you would find it helpful).

General Etiquette & Guidance


I don't know your technical acumen and don't profess to assume, if you are not experienced in this area i would advise refraining from criticism of areas you are not fully knowledgeable on, otherwise if you are knowledgeable of the technology in question (python, pip, venv, pyproject.toml, etc), maybe only point 1 & 3 may be helpful.

Point 1: I.E this is my approach, test hypothesis before making claims(run commands without UV), understand why x technology might be the way it is (pros and cons), if its a bug or has legitimate arguments against it or a better way (up to you how you define that), then its worth opening a bug report or discussion.

Point 2: For example i am a software engineer, but if my LLM throws a hardware error it would be outside my expertise to critisise NVIDIA for how their cuda or firmware, hardware works, without me first doing research to understand what the underlying issue is, and weather its appropriate to raise to NVIDIA (some issues have nothing to do with NVIDIA or the team making cuda or nvidia-open, etc)

Point 3: consequences, if a discussion, PR, issue is opened which is doesn't raise a real issue or point of discussion, it can impact a project maintainers ability to do their work on the project you love/like using, this is a major issue globally recognised with medium to large free/libre software project, maintainers can be heavily burdened by the volume of spam, especially with the rise of transformer models like LLM's, if this gets to a point where its impractical for the maintainer, you can risk them disabling outside contributions all-together (core/approved maintainers only, no outside PR's).

The lead maintainer of CURL (curl andlibcurl), mused about this possibility (of course keeping it open source, but refusing all PR's/patches from non approved/core maintainers), he decided not to, but i understand why he would say that in a bout of frustration.
This since the AI "security bug report" incidents flooded in, where him and the team were unable to quickly identify whether a legitimate security issue exists, or a spam bot sent it (mostly hinging on the fact many legitimate bug reports come from non-native english speakers, so even if its a translation/LLM text, they should consider the merit). you also get people do the same manually (without the LLM writing the report), because the prospect of earning thousands from a bug bounty is tempting (even if it mostly requires security/software experts to find legitimate security issues, CURL is battle tested over decades).


Also if you want to convert a pyproject.toml into a requirements.txt, its pretty trivial in most cases here are two simple methods:

With UV:

uv pip compile pyproject.toml -o requirements.txt

Another way is to use the pip-tools package:

pip-compile pyproject.toml -o requirements.txt

So if you want to introduce this to "fix" the lack of a requirements.txt, you should make a PR.

i personally dont think it would be useful in practice, python has had native pyproject.toml support for a while (it was pretty much feature complete for general use by v3.8, released on October 2019) for this reason compatibility isn't an issue, also because some of these new software/deps wasn't even written when 3.8 was around.

If you decide its important and want to make a PR, i advise to do it in a way that its dynamically generated from the pyproject.toml, this reduces maintainer's burden, you can make a custom pyproject build command/hood or pre-commit hook PR, to keep the requirements.txt updated. its highly unlikely many people will make use of it for this project, but i know sometimes its not about the destination (goal), but the journey (process).

Just thought i would provide some thoughts here in case it inspires/helps you in some way.
(:
Many thanks,
James David Clarke

References:

PEPs for "pyproject.toml" introduced from python 3.7 to 3.8
https://peps.python.org/pep-0621/#dependencies-optional-dependencies

I in LLM stands for intelligence: https://daniel.haxx.se/blog/2024/01/02/the-i-in-llm-stands-for-intelligence/

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment