<__Sander__>
they apparently now changed their internal git attributes, causing my hacks to fail
<__Sander__>
now NPM will try to invoke git again to download packages
jtojnar_ has joined #nixos-dev
jtojnar has quit [Ping timeout: 268 seconds]
jtojnar_ is now known as jtojnar
<Mic92>
__Sander__: do they expose an api to their package manager?
<joepie91>
not a stable one
<joepie91>
last I checked anyway
<joepie91>
basically "yeah, you can use it as a library; but there's no docs and it might change every other version, lol good luck"
jtojnar has quit [Ping timeout: 248 seconds]
jtojnar has joined #nixos-dev
pie_ has quit [Ping timeout: 256 seconds]
disasm has quit [Ping timeout: 260 seconds]
<__Sander__>
Mic92: they do, but their API does not support all functionality to make integration with Nix possible
<__Sander__>
e.g. forcing a package not to be downloaded or obtained from the NPM cache
<__Sander__>
there are no API calls for that :)
jtojnar has quit [Quit: jtojnar]
jtojnar_ has joined #nixos-dev
jtojnar_ is now known as jtojnar
pie_ has joined #nixos-dev
<Profpatsch>
peti: Hm, I made a short excurse into improving yarn2nix, but now I’m back at my purescript 1.11 vs 1.12 Problem.
<Profpatsch>
pulp still needs 1.11 to build itself, but is cool with building 1.12 projects
<Profpatsch>
So we really need both 1.11 and 1.12 with all transitive deps in nixpkgs.
<Profpatsch>
I will try locally generating the hackagePackages with your hint from before, but ultimately this needs to go into nixpkgs I believe.
<Profpatsch>
__Sander__: Oh crap.
<Profpatsch>
Is the package locking in npm sensible?
<Profpatsch>
I was thinking about maybe adding it as another backend to my yarn2nix (in addition to yarn.lock), assuming the information content is similar.
<Profpatsch>
Scopes now work as well, I’ve managed to package a medium-sized project (pulp) with it.
<Profpatsch>
So far there’s no hacks, which I like. But also a lot of TODOs and uncharted node insanity. :P
<Profpatsch>
I noticed yesterday that the project is one year old this month. oO
<__Sander__>
Profpatsch: generally it works but
<__Sander__>
you'll face another problem besides generating Nix expressions
<__Sander__>
you must bypass NPM's content-addressable cache
<__Sander__>
and that can only be done my adding hidden metadata flags to all the package.json files in the node_modules/ folder hierarchy
<__Sander__>
without these properties, NPM will try to consult the content addressable cache
<__Sander__>
which is empty in a Nix builder environment
<Profpatsch>
So they still rely on a cache file to create a node_modules folder from the lockfile?
<__Sander__>
all download go through some magic API
<__Sander__>
that will first consult a cache
<Profpatsch>
Because with yarn2nix I’m not using npm/yarn at all.
<__Sander__>
when the cache has it, then the cached version will be used
<Profpatsch>
I just unpack them with a few lines of bash. :)
<__Sander__>
if the cache does not have it, NPM will download and put it in the cache
<__Sander__>
Profpatsch: how do you deploy packages that have native dependencies
<__Sander__>
?
<__Sander__>
or packages requiring script directives?
<Profpatsch>
Not yet :P
<Profpatsch>
Haven’t yet stumbled on a use for them.
<__Sander__>
also what I noticed is that SHA1 hashes in yarn.lock files can be used directly in Nix expressions
<__Sander__>
with npm package-lock.files you need to convert them
<__Sander__>
they are SHA1 or SHA512
<Profpatsch>
The builder can be overridden like an overlay, so in theory it should be possible to patch misbehaving packages quite fine-grained
<__Sander__>
but in base64 format
<Profpatsch>
Ah, that could be.
<Profpatsch>
Altough that should be a deterministic and pure operation, right?
<__Sander__>
yes
<Profpatsch>
Hm, then I think I’ll stick with yarn.lock only for a while longer still.
<Profpatsch>
I mean it’s not that hard to generate a yarn.lock and I suspect they are going to add support for reading the npm locks as well sometime in the future.
<__Sander__>
hmm that's sounds not really surprising :)
<Profpatsch>
Also I had a lot of “fun” writing the parser, so I don’t want to ruin that. :P
<__Sander__>
heh, the parser or architecting the tool is fun
matthewbauer has joined #nixos-dev
<__Sander__>
what makes me really depressed is finding workarounds for their weird decisions
<__Sander__>
and broken designs
<__Sander__>
such as the fact that the content-addressable cache can't be disabled
<__Sander__>
I also discovered another thing today
<Profpatsch>
Do you parse the "bin"/"directories.bin" stuff in package.json?
<__Sander__>
that makes me really sad
<__Sander__>
Profpatsch: no I don't
<Profpatsch>
The business logic of that is like ten branching decisions and npm does not sanitize these files or check the invariants at all on upload.
<__Sander__>
ah I see
<__Sander__>
yes that's correct
<__Sander__>
dynamic typing, little checking :p
<Profpatsch>
It is forbidden to have both by spec, but lots of packages do it anyway.
<__Sander__>
yes
<Profpatsch>
And npm just doesn’t car.
<__Sander__>
you get undefined behaviour
<__Sander__>
btw
<__Sander__>
I discovered today that
<__Sander__>
NPM packages with optionalDependencies
<Profpatsch>
“It works for most cases ship it. Oh, now we have undefined behaviour. Let’s just make it work anyhow“
<__Sander__>
if you publish them to the NPM registry
<__Sander__>
the optionalDependencies get added to the "regular" dependencies
<__Sander__>
my generator directly uses the "dependencies" field from the NPM registry
<Profpatsch>
Ouch, I see.
<__Sander__>
and suddenly optional dependencies pollute my builds causing it to fail
<__Sander__>
I have to write something now that filters out the optionalDependencies from the regular dependencies
<__Sander__>
I think this is a hack made by the NPM developers
<__Sander__>
why having an optionalDependencies field, and secretly adding these packages to the regular dependencies as well? it makes no sense to me :p
lopsided98 has quit [Ping timeout: 260 seconds]
lopsided98 has joined #nixos-dev
__Sander__ has quit [Quit: Konversation terminated!]
matthewbauer has quit [Remote host closed the connection]
matthewbauer has joined #nixos-dev
pie_ has quit [Ping timeout: 256 seconds]
matthewbauer has quit [Remote host closed the connection]
orivej has joined #nixos-dev
matthewbauer has joined #nixos-dev
Sonarpulse has quit [Ping timeout: 276 seconds]
Sonarpulse has joined #nixos-dev
matthewbauer has quit [Read error: Connection reset by peer]
matthewbauer has joined #nixos-dev
orivej has quit [Ping timeout: 245 seconds]
jtojnar has quit [Quit: jtojnar]
jtojnar has joined #nixos-dev
{^_^} has quit [Remote host closed the connection]
{^_^} has joined #nixos-dev
{^_^} has joined #nixos-dev
{^_^} has quit [Remote host closed the connection]
{^_^} has joined #nixos-dev
{^_^} has joined #nixos-dev
matthewbauer has quit [Read error: Connection reset by peer]
matthewbauer has joined #nixos-dev
<LnL>
does anybody know what's up with all of the aborted jobs on staging?
matthewbauer has quit [Ping timeout: 245 seconds]
pie_ has joined #nixos-dev
matthewbauer has joined #nixos-dev
goibhniu has joined #nixos-dev
matthewbauer has quit [Ping timeout: 264 seconds]
jtojnar has quit [Read error: Connection reset by peer]