worldofpeace changed the topic of #nixos-dev to: NixOS Development (#nixos for questions) | NixOS 20.09 Nightingale ✨ | | | 20.09 RMs: worldofpeace, jonringer |
zarco has quit [Ping timeout: 260 seconds]
zarco has joined #nixos-dev
sorear has quit [Ping timeout: 260 seconds]
sorear has joined #nixos-dev
supersandro2000 has quit [Disconnected by services]
supersandro2000 has joined #nixos-dev
alp has quit [Ping timeout: 272 seconds]
cole-h has quit [Ping timeout: 260 seconds]
ris has quit [Ping timeout: 256 seconds]
v0|d has quit [Remote host closed the connection]
v0|d has joined #nixos-dev
justan0theruser has quit [Quit: WeeChat 2.9]
justanotheruser has joined #nixos-dev
<ivan> at least your new karma is prime
evils has quit [Quit: Lost terminal]
stoile has quit [Ping timeout: 268 seconds]
stoile has joined #nixos-dev
evils has joined #nixos-dev
orivej has quit [Ping timeout: 256 seconds]
supersandro2000 has quit [Quit: The Lounge -]
supersandro2000 has joined #nixos-dev
jonringer has joined #nixos-dev
orivej has joined #nixos-dev
<etu> ✨ ryantm
<{^_^}> ryantm's karma got increased to 25
<etu> (we're getting updates of php-packages)
<siraben> Can anyone mind taking a look at ?
<{^_^}> #102503 (by siraben, 1 day ago, open): Initial implementation of remarkable1 cross-compile
orivej has quit [Ping timeout: 260 seconds]
luc65r has joined #nixos-dev
cole-h has joined #nixos-dev
supersandro2000 has quit [Quit: The Lounge -]
supersandro2000 has joined #nixos-dev
alp has joined #nixos-dev
saschagrunert has joined #nixos-dev
cransom has quit [Ping timeout: 260 seconds]
supersandro2000 has quit [Quit: The Lounge -]
supersandro2000 has joined #nixos-dev
cole-h has quit [Ping timeout: 264 seconds]
FRidh has joined #nixos-dev
cransom has joined #nixos-dev
jonringer has quit [Ping timeout: 264 seconds]
alp has quit [Remote host closed the connection]
alp has joined #nixos-dev
zowoq[m] has quit [Quit: Idle for 30+ days]
<roberth> siraben: can't promise anything, but perhaps you could also ask in #nixos-aarch64?
<siraben> roberth: Ok, how come there?
<roberth> siraben: well it's not exactly the right topic, but possibly some of the right people
<siraben> Thanks.
<roberth> if anyone here has experience with setting up cross compilation for arm hardware platforms or similar,
<{^_^}> #102503 (by siraben, 1 day ago, open): Initial implementation of remarkable1 cross-compile
__monty__ has joined #nixos-dev
sphalerite has quit [Ping timeout: 260 seconds]
das_j has quit [Ping timeout: 272 seconds]
immae has quit [Ping timeout: 272 seconds]
hexa- has quit [Ping timeout: 272 seconds]
yorick has quit [Ping timeout: 272 seconds]
immae has joined #nixos-dev
sphalerite has joined #nixos-dev
das_j has joined #nixos-dev
yorick has joined #nixos-dev
hexa- has joined #nixos-dev
alp has quit [Ping timeout: 272 seconds]
alp has joined #nixos-dev
rajivr has joined #nixos-dev
alp has quit [Ping timeout: 272 seconds]
alp has joined #nixos-dev
FRidh has quit [Remote host closed the connection]
alp has quit [Remote host closed the connection]
alp has joined #nixos-dev
FRidh has joined #nixos-dev
<eyJhb> How would we fix such a thing here -
<eyJhb> Not apply the patch to that revision? Or patch the patch?
orivej has joined #nixos-dev
<zimbatm> FRidh: I am looking into maping python packages 1:1:1 mapping between PyPi, attribute and pname
<zimbatm> would you be open for me to change the organization of the python packages quite drastically?
<zimbatm> we will need an alias map for the renames at minimum
<zimbatm> and I was thinking of moving pkgs/top-level/python-packages.nix to pkgs/development/python-modules/default.nix
<zimbatm> just to co-locate all of that stuff
<FRidh> zimbatm: attributes should correspond to pypi names. When that isn't the case, its a "bug". pname, ideally as well, but consider that pname is typically inherited by fetchPypi (better to have it the other way around to follow the dependency tree).
<FRidh> the hard part is to keep it like that
<FRidh> also, regarding the mapping, it is important the names are normalized, so lowercase and with a dash instead of underscore or dot
<zimbatm> cool, at least we agree on principle
<FRidh> If you want to make these changes, please open an issue. We then need to find a time to make it because it will conflict with a lot of other open PRs
<zimbatm> I wrote a script that checks for the mapping right now
<zimbatm> maybe I can start with posting that report
<FRidh> Right
<zimbatm> or do you have one of your own?
<FRidh> no I don't
<zimbatm> FRidh: why does the attribure name need to be lowercased?
<zimbatm> same question about the dashes
<zimbatm> I understand for example that "." is the package name needs to be replaced with something else
<FRidh> this is the normalized name as used in pep 503.
<zimbatm> I see
<zimbatm> isn't that the job of PyPi to enforce that?
<FRidh> these characters are irrelevant for certain things. However, they are relevant for the name of the files downloaded, which is annoying, because a project can call its project Foo and then an artifact foo
<zimbatm> for us, I think it's more important to map 1:1 to make packages lookups more predictable
<FRidh> that's the thing, you cannot
<zimbatm> as a developer, I often first look things up on PyPi and then want to be able to copy the name into Nix
<zimbatm> without the 1:1 mapping it means that I have to build and hope that it matches
<FRidh> which won't work because of the dot as you mentioned. Also, pypi names can start with a number
<FRidh> you can normalize the pypi name using this simple rule
<FRidh> only exception is still projects starting with a digit
<V> nix-repl> { "3" = 4; }
<V> { "3" = 4; }
<V> where's the problem with that
<V> callPackage?
<zimbatm> hmm
<zimbatm> I should have known better for this to be simple
<FRidh> haha
<FRidh> so many of these things would have been looking much better if they were simple
<FRidh> I'm finally working on separating propagatedBuildInputs from the wrappers,
<{^_^}> #102613 (by FRidh, 1 day ago, open): WIP: Python: wrap and patch using `requiredPythonModules` instead of `propagatedBuildInputs`
<FRidh> I don't want to know how much that is going to break...
<zimbatm> I started writing a RFC to allow abitrary keys in nix to better support numbered attributes
<FRidh> V: `let x = {"3foo" = "bar";}; in with x; "3foo" now you need the double quotes, wheres normally with packages we never have to
<FRidh> though I recall haskell doing it like that, but I may be wrong
<V> node also does that
<V> hence I'm not sure that's an issue
<FRidh> if its consistent, no
<FRidh> if its some packages, yes
<zimbatm> the main issue is callPackage and accessing the package with the CLI through `-A`
<V> yep, callPackage totally breaks
<FRidh> callPackage as well? did not know that
<V> well, how do you quote a name in function parameters?
<zimbatm> `{ 3 }: 3`
<zimbatm> it would be possible if the Nix language was extended
<FRidh> right
<V> it would be possible if we stopped using callPackage :}
<zimbatm> `{ "3"@alias }: alias`
<FRidh> but that's a bit out of scope here
<gchristensen> redefining 3 would be fascinating
<zimbatm> yeah
<zimbatm> ^ but basically you would have a lookup key, and then have to generate an alias for it
<V> so, what node has is a let ... in, where all the libraries go in an attrset in the let, and the binaries all refer to that attrset
<V> I think that's okay
<FRidh> so you call it also with the set then?
<V> hmm? the binary packages have `dependencies = [ sources."@angular-devkit/architect-0.10001.7" sources."@angular-devkit/core-10.1.7" ... ];`
<V> and so on
<FRidh> yes, and such an expression is then in the form of { sources} : ... ?
<V> no, it's let sources = { (sources go here) }; in { (programs go here) }
<FRidh> yes, checking one such file now, its all within the same file
<V> ofc that could be adapted to multiple files
<zimbatm> regarding -A, I just tested and `nix-build -A '""'` works. Where "" is the attribute name
<FRidh> right. At which point you would use e.g. callPackage and pass in `sources`, so the set
<FRidh> zimbatm: which requires users to know that they need to use the extra quotes
<zimbatm> yes but it's possible at least
<FRidh> but good to know :)
<siraben> Hi, can anyone help test ? It adds MMIX (yes the CPU in Kunth's The Art of Computer Programming) as a cross compilation target
<{^_^}> #102766 (by siraben, 1 minute ago, open): Initial implementation of cross-compilation of Knuth's MMIX
<zimbatm> it's mostly useful for tooling
<zimbatm> I can see us generating 1:1 maps, and then maintaining the nix aliases as well
<V> FRidh: to be fair, any user-installed package is somewhat unlikely to start with a number
<V> indeed, most packages are unlikely to
<zimbatm> unfortunately somebody decided to name their tool "1password"
<V> but that's kind of a limitation of Nix :/
<zimbatm> yeah that's alright
<V> that is true
<V> and also 0ad
<V> and 010 editor
<FRidh> I think in python-packages.nix we prefix now such packages with `py`
<FRidh> although, of course, that could lead to a collision
<V> I would opt to remove such prefixing for consistency
<FRidh> you need something, because otherwise it means using quotes
<V> I think that quotes are okay
<zimbatm> and callPackage is useful
<V> you could prefix everything with an underscore
<zimbatm> I'd rather all the dependencies being explicitly named
<V> like, absolutely everything
<FRidh> i'd rather python packaging to be sane
<V> or alternatively, just prefix numbers and underscore-prefixed packages with underscore
<FRidh> yes, probably the best option
<V> it's not pretty, but it's also there because we're working around the more fundamental issue of callPackage
<zimbatm> that's what we do in the top-level
<V> which itself is poorly-designed
<zimbatm> but it doesn't work for packages with dots (.) in them
<FRidh> we do?
<V> we do?
<V> I thought 0ad was zeroad
<zimbatm> _1password
<zimbatm> _9pfs
<V> well, that's about as consistent as nixpkgs usually is
<zimbatm> we have all the data, it's just a matter of enforcing it with CI
<V> yup. zeroad = zeroadPackages.zeroad
<zimbatm> I will add it once the manual is converted to markdown :p
<FRidh> I sometimes check the nix repo for doc changes, not many yet
<V> > The name attribute must not contain uppercase letters — e.g., "mplayer-1.0rc2" instead of "MPlayer-1.0rc2".
<{^_^}> error: syntax error, unexpected $undefined, expecting ')', at (string):345:55
<V> that certainly isn't applied, e.g. in perl or C#
<FRidh> right, we have that one as well for the attribute name zimbatm
<FRidh> be careful what you get yourself into :p
<V> me? I've already seen this before >.>
<roberth> callPackage again? I haven't heard a convincing argument for it
teto has joined #nixos-dev
<zimbatm> doesn't callPackage also wrap the derivation to make it overridable?
<V> see the following comment
<FRidh> and implements splicing for cross
<zimbatm> for most packages, we could have something simple
<zimbatm> pkgs/simple/<attr>
orivej has quit [Ping timeout: 264 seconds]
<zimbatm> where <attr> is both the folder name and attribute name
<V> ah, bazel
<FRidh> if you want to get rid of callPackage, its better to revisite the whole way we describe packages to begin with
<V> yeah
<FRidh> won't happen in nixpkgs though
<zimbatm> it's one of the best thing of bazel
<V> it is
<zimbatm> I'm tired of constantly looking things up
<zimbatm> obviously there are special cases where that doesn't work
<zimbatm> but at least we could have something simple by default
<zimbatm> the only issue with that approach is that the whole tree is loaded on each evaluation
<V> I don't remember whether it is
<V> I guess it's a necessity of the way the expression language works
<zimbatm> the function the attrset, which is generated recursively
<V> you could probably work around that with functors
<zimbatm> I mean the function returnns the attrset
lukegb has quit [Ping timeout: 265 seconds]
<roberth> the idea is to remove the intersectAttrs from callPackage. I know it does the overriding stuff
<roberth> without the intersectAttrs you could do pypkgs@{...}: ..... pypkgs."3foo"
<zimbatm> wouldn't that require to rewrite the whole tree and add `, ...` to all the function definitions
<roberth> that'd be a more honest way to write it in the first place
<V> anything removing intersectAttrs would require that
<V> I'm not sure about ...
<zimbatm> we need more AST-rewriting tools in general
<zimbatm> it would be easy to try
<roberth> ... seems to have a bad rep for no good reason and no-one knows why in that thread
<V> it has a bad rep because it hides mistakes
<roberth> not in the case of callPackage though
<V> when you think you're passing in a specific attribute but you typo'd it, for instance
<V> I've definitely had that happen to me
<roberth> that's not how callPackage is used and it prevents pypkgs."3foo"
<zimbatm> we write `callPackage` so we don't have to write `, ...`. lol
<roberth> in other code it's better to omit if not necessary of course
<V> if nixpkgs were tree-structured, would there be anything wrong with just pkgs:
<roberth> not sure what you mean with tree-structured
<V> like with bazel
<V> in bazel you have //foo/bar/baz, where foo, bar, and baz are directories, and // refers to the root
<zimbatm> pkgs.development.python-modules.pyOpenSSL
<V> (simplified down)
<V> it's easy to find everything, everything has a predictable name
<zimbatm> maybe it would force us to not have overly-deep folder structures
<FRidh> many of the folder structures are not maintained
<roberth> I don't think it has to be tree-structured for pkgs: to work
<V> it doesn't, that was just an example though
<FRidh> anyway, do consider variants of packages and package sets
<zimbatm> one of the nice thing about callPackage is the ability to create a new scope
<zimbatm> I would that quite useful a number of times
<zimbatm> ^
<roberth> sorry, I should have clarified immediately that I don't want to remove callPackage, just simplify it to remove intersectAttrs
<gchristensen> another thing is it is useful to reduce the amount of nixpkgs it has to keep in memory
<V> zimbatm: scopes can bite you in the ass
<V> take pam, for instance
<zimbatm> roberth: yeah, I was just thinking out loud
<zimbatm> indeed
<roberth> gchristensen: `pkgs` or `self` or whatnot is always in memory. `intersectAttrs` only causes more allocations
<V> also consider the much-needed addition of package configuration
<V> currently it's very ad-hoc and will break if someone adds a package to nixpkgs that has the same name as your config option
<zimbatm> now if we consider that pkgs is re-structured to map the folder hierarchy 1:1, then the argument injection becomes much less useful
<V> which necessitates refactoring the structure of a package definition regardless
<V> zimbatm: yep
<zimbatm> { stdenv, build-support, development }: stdenv.mkDerivation { src = build-support.fetchFromGitHub { ... }; }
<zimbatm> but I think we can afford a two-layer system
<zimbatm> where the complicated stuff can stay in the old style
<FRidh> that separation is a bit arbitrary
<zimbatm> for sure
<roberth> pattern matching on `pkgs` does give you one layer of scope checking
<zimbatm> we can copy arch to steal the list of "core" package set
<V> don't forget gentoo :)
<zimbatm> gentoo has a much larger core isn't it?
<V> yes, but it's also somewhat more intelligently laid out IIRC
<V> as far as categorisation is concerned
<zimbatm> pkgsrc is not too bad either
<gchristensen> arch and gentoo's package manager has late binding
<V> it's a shame how nix is not lazy in one of the most useful ways it could be
<gchristensen> I disagree
<ryantm> @etu yay!
<V> gchristensen: mind telling why? I'm genuinely curious
<gchristensen> having access to thederivation build instructions adds a model of capabilities which, imo, makes Nix possible
<V> I'm not sure I see why you have to lose objcap here?
<gchristensen> maybe not
<V> nothing would change except for derivations that aren't ever needed being not loaded
<gchristensen> maybe I misunderstood what you mean
<gchristensen> t
<V> perhaps
<V> to be fair IRC perhaps isn't the best medium for stuff like this
<gchristensen> hehe
Baughn has quit [Read error: Connection reset by peer]
Baughn has joined #nixos-dev
bennofs[m] has quit [Quit: Idle for 30+ days]
<siraben> Ericson2314: let me know when you can review , this one was quite simple to add and works well
<{^_^}> #102766 (by siraben, 1 hour ago, open): Initial implementation of cross-compilation to Knuth's MMIX
orivej has joined #nixos-dev
lukegb has joined #nixos-dev
cole-h has joined #nixos-dev
jonringer has joined #nixos-dev
<Taneb> ...looking at nixpkgs, I think the "boot.zfs.forceImportAll" doesn't do anything
<V> plausibly not
<V> yup
<V> it's only used in an assertion, and nowhere else.
<Taneb> Unless there's some magic going on where I'm not looking
<V> I doubt it
<V> I've found dead attributes before
<V> maybe take a peek at git blame?
<Taneb> Looks like it died between 18.03 and 18.09
<{^_^}> #42269 (by Baughn, 2 years ago, merged): zfs: Improve import handling
FRidh has quit [Quit: Konversation terminated!]
alp has quit [Ping timeout: 272 seconds]
alp has joined #nixos-dev
rajivr has quit [Quit: Connection closed for inactivity]
jonringer has quit [Remote host closed the connection]
alp has quit [Ping timeout: 272 seconds]
justanotheruser has quit [Ping timeout: 272 seconds]
ris has joined #nixos-dev
ris has quit [Ping timeout: 256 seconds]
saschagrunert has quit [Remote host closed the connection]
ris has joined #nixos-dev
<samueldr> I think I asked previously, but I'm wondering if, without configuring hooks, it would be worthwhile to allow derivations to opt-out of sandbox paths
<samueldr> main thing I have in mind is the ambient /bin/sh
<samueldr> (which on NixOS comes from a specific busybox configuration build)
<samueldr> while debatable as we could say "it's the posix interface", it _is_ somewhat of an impurity
justanotheruser has joined #nixos-dev
<samueldr> It feels a bit icky that it's not part of the hashed inputs
v0|d has quit [Read error: Connection reset by peer]
<samueldr> Disabling all sandbox paths in NixOS makes some things fail, if they were not built beforehand
<gchristensen> I'm in to it
<LnL> I would love it if we got rid of that
<gchristensen> tangentially I bet there is a way to use linker scripts to replace symbolic program paths with materialized paths, and I wish it was a thing
<samueldr> Would it make sense to have opt-in hashed sandbox exceptions?
<samueldr> So that /bin/sh in Nixpkgs becomes part of the hash
<samueldr> But... Backward compatibility
__monty__ has quit [Quit: leaving]
<gchristensen> it'd be foolish to have my machine connect to machine A for a remote build, and that machine A's nix-daemon connects to machine B to actually do the remote build right?
<samueldr> why?
<samueldr> why would you care what the remote builder does?
<gchristensen> well... sounds fragile :)
<LnL> I send all my builds through the daemon which is basically the same thing but locally
alp has joined #nixos-dev
zowoq[m] has joined #nixos-dev
luc65r has quit [Quit: WeeChat 2.9]
supersandro2000 has quit [Ping timeout: 265 seconds]
alp has quit [Ping timeout: 272 seconds]
supersandro2000 has joined #nixos-dev