<qyliss>
presumably worry about stepping on peti's toes because nobody else would maintain haskellPackages?
<hexa->
maralorn: ^
LnL- is now known as LnL
<ekleog>
https://github.com/NixOS/rfcs/pull/72 hopefully when we implement the switch to markdown it'll make it easier to get everything together again
<{^_^}>
rfcs#72 (by mboes, 23 weeks ago, merged): [RFC 0072] Switch to CommonMark for documentation
dstzd has joined #nixos-dev
ghuntley has quit [Ping timeout: 244 seconds]
taktoa[c] has quit [Ping timeout: 244 seconds]
ris has quit [Remote host closed the connection]
ris has joined #nixos-dev
taktoa[c] has joined #nixos-dev
<qyliss>
Is that why the docs were moved?
ghuntley has joined #nixos-dev
<qyliss>
ekleog: actually it can't be, because the haskell docs were already in Markdown
<ekleog>
hmmm then I'm at a loss as to why this move happened
<andi->
IIRC as the TOC didn't support sufficient enough levels of sub-categories
<qyliss>
andi-: seriously?
<gchristensen>
fixing it meant fixing docbook
<gchristensen>
I'd imagine when we're .md'd he'd come back
<andi->
that is what I think maralorn told me.. another reason was missing links to href's but as per my investigation all of them were there and the markup looked brokne. It was an empty <a></a> tag with the right href..
<infinisil>
Iirc it was because not all headers were displayed in the TOC anymore
<infinisil>
They were previously, but somebody changed it to only show the first two levels
<infinisil>
And I don't think peti liked that
<qyliss>
gchristensen: was that discussed somewhere linkable?
<maralorn>
Also peti was annoyed about the fact, that the restructuring broke the deeplinks into the docu.
<gchristensen>
no specific thing in mind just memory
<maralorn>
So he was like, „I’m gonna do it on my own, because I don‘t trust them with that anymore.“
<hexa->
probably talked about on haskell fridays on discord?
<maralorn>
Of course an in depth.
<maralorn>
I was against it.
<qyliss>
it's concerning to me that the docs were moved just because one person didn't like how they were rendered
<maralorn>
I announced it here, but the reactions were weak to none existant.
<maralorn>
qyliss: to be fair he wrote a lot of it.
<maralorn>
And the rendering is bad.
<maralorn>
Or let's say improvable.
<maralorn>
My hope is that, the manuel rendering will improve and the migrating the haskell documentation back will be an option.
<infinisil>
As much as I should appreciate peti maintaining haskell packages, I don't think he has any intention of improving the state of it
<maralorn>
infinisil: The state of the documentation or of the package set?
<infinisil>
The package set mainly
<infinisil>
Like, as far as I know, he's been adding manual overrides to packages for a long time now, but seems to be of the opinion that this is the right way of doing it and has no intention of changing anything
<maralorn>
infinisil: peti and I are at least talking about a pretty big restructuring and improvement. So the intention is definetely there. peti wanted to write up a proposal for the changes and the new tooling.
<infinisil>
Oh that's nice to hear
<infinisil>
maralorn: What's the restructuring look like?
<infinisil>
Or better yet, is there any public discussion of this?
<maralorn>
infinisil: All of it has been public in the sense, that it happened on a public twitch stream. The idea was to write something initial up and then invite interested people to a call or something.
<maralorn>
The biggest issue I see right now is that it's quite a lot of work and only peti and me are invested in it and we don‘t really have any spare time …
<infinisil>
maralorn: Can you summarize the idea briefly?
<maralorn>
infinisil: We have 2 main concerns a) we want to make enabling and disabling haskellPackages much easier and b) we consider including packages on opt-in basis as opposed to opt-out as it is now. To only have packages enabled when they have a "maintainer". Then when something breaks we have someone we can ask if they or someone should fix the package or if we can drop it.
<maralorn>
Right now we maintain 6000 packages with roughly 3 people and it's hard to tell which of those are important and which not.
<maralorn>
Most arguments for that change are not directly from a user perspective. 1. We do a lot of grunt work every week for packages which probably no one uses. 2. Right now improvements in cabal2nix are a bit capped by the fact that any change that introduces more lines will make the hackage-packages.nix explode.
<infinisil>
I mean, the current approach mostly works. What if we just enabled doJailbreak and dontCheck by default for all packages?
<qyliss>
could the latter be solved by just splitting it up into more files?
<infinisil>
That should cut maintenance by a lot
<maralorn>
infinisil: I guess that would help. But on the other hand we would loose stability and quality. And I can think of quite a few regular issues that wouldn‘t disappear by it.
<infinisil>
Afaik, if currently a package fails either due to constraints or test failures, people just add doJailbreak/dontCheck without any consideration about whether this causes any problems
<maralorn>
My position on this is: We should do this only when we have tooling that makes it dead-simple to activate additional packages.
<infinisil>
I like the opt-in idea, but I think it would be way too detrimental to make all packages opt-in
<infinisil>
How about just making the ones that can't automatically by fixed opt-in
<maralorn>
infinisil: No, quite a lot of times doJailbreak just gives you build error which is harder to understand. We regularly encourage people to report jailbreak issues upstream, for cabal files with flags jailbreak doesn‘t even work.
<infinisil>
Yeah, I mean cases where doJailbreak/dontCheck makes it work without problems
<maralorn>
infinisil: Yeah, that is a cool idea.
<infinisil>
CI tries to build -> it fails, adds doJailbreak -> works, alright let's apply that
<maralorn>
That would cost a lot of build time.
<infinisil>
Yeah so maybe just skip the first one, and just build them all with doJailbreak enabled
<infinisil>
Maybe we should just measure how many packages newly succeed with this
<maralorn>
The problem with that approach is, that users never know if a package builds by accident or because we care about it.
<maralorn>
So we could just drop it at any time.
<infinisil>
Is that a problem?
<maralorn>
For me as a maintainer it feels like a problem.
<infinisil>
With Haskell especially, if it builds there's a very high chance it works correctly
<maralorn>
Well by this approach we basically say "cabal upper version bounds are useless"
<maralorn>
tbh I am not even sure where I stand on this.
<maralorn>
I‘d like to have a well-informed discussion on this. But in the end we need a solution that actually some implements.
<infinisil>
Yeah, imo they are pretty useless, I've never seen a case where they were used to prevent some non-compile error
<infinisil>
That's one good thing about the doJailbreak/dontCheck by default: It should be relatively easy to implement
<maralorn>
Right now our incentives are "Let's finde the mode which is the easiest to maintain."
<infinisil>
Well if that's the case, we should remove the haskell set entirely! No package set, no maintenance :)
<maralorn>
And having a lot more packages enabled, I don‘t know if that's less work at the end of the day …
<infinisil>
You also need to have another objective to balance it out
<maralorn>
Fair
<maralorn>
But I gotta go now, sorry.
<infinisil>
Np :)
<infinisil>
Thanks for the info
andi- has quit [Remote host closed the connection]
<Profpatsch>
Ericson2314: Is it about keeping the intermediate build results off of the local machine though?
<Ericson2314>
hit 'em up
<Ericson2314>
Profpatsch: yes
<Profpatsch>
whoa that’s amazing
<Profpatsch>
I’ve been waiting for somebody to finally fix the scheduler
<Ericson2314>
that's just a biproduct
<Profpatsch>
Usually good refactors lead to good results just by happenstance
<Ericson2314>
it's just schedule as the `--store`, pass `--builder 'auto - - 1 1'` to do local machines
<Ericson2314>
Profpatsch: we'll it's kind of a nasty non-refactor with the downcasting :D
<Profpatsch>
Because you get a better view on the code and notice stuff that is not optimal because it starts to stand out
<Ericson2314>
but yeah it's the right idea
<halfbit>
gcc builds seems to be pretty convoluted with the default builder.sh
<halfbit>
like I was just trying to figure out why I always got -debug builds for cross toolchains
<halfbit>
and even that was a mystery, the commentary surrounding why -g is used in that situation is super unclear, git history didn't help much
<Ericson2314>
halfbit: ignore our default builder.sh if you want to pitch the project except to say "well, our current bidl is also over complex, so that isn't a downside of doing the granular approach"
<halfbit>
so the granular approach is build a statically linked compiler (no shared libs needed) that then builds libc, and libgcc+friends?
<halfbit>
if I understand that right?
<halfbit>
and then... eventually builds a gcc that uses those libs itself?
<Ericson2314>
Profpatsch I do hope the change does increase the impetus to do the second split (build.cc -> derivation-goal.cc -> ??)
<Ericson2314>
while also "proving" that much of that code really is about scheduling not building
<Ericson2314>
halfbit: you just build one *compiler*, but multiple *toolchains*
<Profpatsch>
Ericson2314: I’d love to see a formalization of the builder, or at least a good design document
<Ericson2314>
Profpatsch: yes, I need the final refactor to happen so I know what the hell I'm doing in hnix-store :D
<Profpatsch>
My goal would be to never build anything locally and to reduce the amount of networking to a minimum
<Ericson2314>
Profpatsch: and never *push* data to the other side
<Ericson2314>
let them pull from you
<Ericson2314>
that's the real nice bit
<Profpatsch>
Ericson2314: ideally I wouldn’t even want the source to be a local file
<Ericson2314>
nix source or other source?
<Profpatsch>
The source if I’m building from a local repository
<Ericson2314>
Profpatsch: if I get back to the IPFS nix stuff, I want to see private swarm just move all the content-addressed data around, "slosh back in forth", while a damn-near completely separate things schedulings building
<Profpatsch>
I don’t want to have to fetch the whole source dir, just the files I need directly from the git remote
<Profpatsch>
Ideally nothing hits the CI node
<Ericson2314>
oh yeah we should have CA-derivation versions of all builtin.fetch and builtin.filterSource
<Ericson2314>
and be able to walk path in merkle tree not downloading siblings
<Profpatsch>
but also, like, if I build something at the moment I have to reference actual files
<Ericson2314>
so you should always be signed-thing-relative
<Profpatsch>
Maybe we are talking about a slightly different thing here, but sgtm :)
<Profpatsch>
I guess even some kind of fuse that insta-mounts would be good.
<Ericson2314>
yeah same
<Ericson2314>
cross the fuse bind-mount divide
<Ericson2314>
Profpatsch: last thing is in the other thread, for uploading sake, it would be nice to have "layered stsore" so just upload the things that cache.nixos.org doesn't already have
<Ericson2314>
take that, Docker :P
<Profpatsch>
Ericson2314: I think you can pretty much achieve that with post-build-hooks and a second substituter
<Profpatsch>
I don’t know how nix behaves though and if it tries one after the other or both at the same time
<Profpatsch>
We have that on CI
<Ericson2314>
fair enough :)
<qyliss>
I think it tries them sequentially
<qyliss>
because I remember seeing an issue about changing it to the other
<gchristensen>
yea
<gchristensen>
it is also why having multiple caches (say, many w/ cachix) is very slow