gchristensen changed the topic of #nixos-dev to: NixOS Development (#nixos for questions) | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html | 18.03 release managers: fpletz and vcunat
jtojnar_ has joined #nixos-dev
jtojnar has quit [Ping timeout: 268 seconds]
jtojnar_ is now known as jtojnar
coconnor has joined #nixos-dev
xeji has quit [Quit: WeeChat 2.0]
lopsided98 has quit [Remote host closed the connection]
lopsided98 has joined #nixos-dev
lopsided98 has quit [Remote host closed the connection]
lopsided98 has joined #nixos-dev
<shlevy> simpson: A bit off-topic, but I've been told to ping you :) https://twitter.com/grhmc/status/979163274670460929 Able to join #nixos-chat?
<simpson> shlevy: For distributed stuff, take a look at Capn Proto's builtin RPC, which can represent capabilities on-the-wire in a pretty powerful way; it's based on E's CapTP protocol. For single-system stuff, if you can read Java then http://www.erights.org/talks/promises/paper/tgc05-submitted.pdf is a classic.
<simpson> I'll idle in #nixos-chat but you can also ask in #erights about this sort of stuff.
<shlevy> Cool, thanks!
mbrgm has quit [Ping timeout: 240 seconds]
mbrgm has joined #nixos-dev
<thoughtpolice> Ugh, JDK 10 requires either a pre-existing JDK 10 or a JDK 9 install. But JDK 9 is EOL'd. :(
<thoughtpolice> Maybe I should just not expose openjdk9 as an attribute but keep the expression as the boot JDK? Any thoughts?
<thoughtpolice> Ah, there's a bootstrap.nix...
<thoughtpolice> lol they all point to pre-existing dropbox URLs
<thoughtpolice> Ah, it's shlevy's dropbox :P
ma27 has quit [Ping timeout: 246 seconds]
jtojnar_ has joined #nixos-dev
jtojnar has quit [Ping timeout: 248 seconds]
jtojnar_ is now known as jtojnar
<coconnor> is there a mechanism to introduce an ipfs mirror of urls?
<coconnor> (yea yea. ipfs:// is a url ya know what I mean)
<coconnor> is there a mechanism to say something like "Here is a HTTP URL for this or try the IPFS"
orivej has joined #nixos-dev
phreedom has quit [Read error: Connection reset by peer]
phreedom has joined #nixos-dev
zybell_ has quit [Ping timeout: 264 seconds]
orivej has quit [Ping timeout: 260 seconds]
jtojnar has quit [Read error: Connection reset by peer]
davidlt_ has joined #nixos-dev
phreedom has quit [Ping timeout: 268 seconds]
phreedom has joined #nixos-dev
MichaelRaskin has quit [Quit: MichaelRaskin]
JosW has joined #nixos-dev
JosW has quit [Read error: Connection reset by peer]
goibhniu has joined #nixos-dev
__Sander__ has joined #nixos-dev
jtojnar has joined #nixos-dev
davidlt__ has joined #nixos-dev
davidlt_ has quit [Ping timeout: 264 seconds]
davidlt__ has quit [Remote host closed the connection]
davidlt__ has joined #nixos-dev
davidlt__ is now known as davidlt
ma27 has joined #nixos-dev
pie__ has quit [Ping timeout: 240 seconds]
davidlt has quit [Remote host closed the connection]
davidlt has joined #nixos-dev
JosW has joined #nixos-dev
<ikwildrpepper> looks like switch-to-configuration in 18.03 is returning a non-zero exit code (4) on upgrade from 17.09. has anyone else seen this?
<ikwildrpepper> I see some lines with "********.service is not active, cannot reload."
<ikwildrpepper> e.g. dbus.service is not active, cannot reload.
<domenkozar> yeah I've seen that as well
<domenkozar> on all upgrades
<domenkozar> (3)
<ikwildrpepper> domenkozar: thnx
coconnor has quit [Ping timeout: 264 seconds]
ma27 has quit [Ping timeout: 276 seconds]
ma27 has joined #nixos-dev
ma27 has quit [Ping timeout: 276 seconds]
orivej has joined #nixos-dev
JosW has quit [Quit: Konversation terminated!]
ma27 has joined #nixos-dev
Synthetica has joined #nixos-dev
pie__ has joined #nixos-dev
pie__ has quit [Ping timeout: 248 seconds]
<dtz> not sure what happened but upgrading to 18.03 from 17.09 on my builders resulting in them ..going offline? but worked after forced them to reboot in person :/
<domenkozar> quite a few people reported such issues
<ikwildrpepper> shouldn't that be a blocker? :)
<ikwildrpepper> dtz: were these physical machines?
<ikwildrpepper> I saw the same behavior with VirtualBox this morning
jtojnar_ has joined #nixos-dev
jtojnar_ has quit [Remote host closed the connection]
<domenkozar> fpletz: ^^
<dtz> they were physical machines
<dtz> sorry for not getting more info, they're not easy to hook up a monitor/keyboard to O:)
jtojnar has quit [Ping timeout: 268 seconds]
ma27 has quit [Ping timeout: 256 seconds]
txp284 has joined #nixos-dev
zybell has joined #nixos-dev
<dtz> thoughtpolice: maybe silly q but re:ubuntu 16.04-- sandbox works? O:)
txp284 has quit [Client Quit]
abbradar has joined #nixos-dev
pie__ has joined #nixos-dev
pie__ has quit [Ping timeout: 256 seconds]
<dtz> shlevy: fixed link, sorry! Will try to answer your q in a bit
<thoughtpolice> dtz: Yeah, we do use it -- should work.
pie_ has joined #nixos-dev
pie_ has quit [Ping timeout: 256 seconds]
<gchristensen> I'd like to rename "Multi-User" to "Nix-Daemon" in the installer, to eliminate the "but I'm the only user" complaint
<gchristensen> any opinions?
<domenkozar> +1
<thoughtpolice> +1
<abbradar> +1, I had questions about that
<gchristensen> +3 is more than -1 so I'm going to take that as a resounding yes
<globin> +1
<gchristensen> =D
<shlevy> +1
<globin> niksnut: could you check if there are any errors in the website deployment, packages-unstable.json.gz is not updating
<domenkozar> globin: btw did you see we have a lot people reporting issues on upgrade?
<domenkozar> with activation exiting with non-zero
<gchristensen> I wonder if nixos.org is using nix 1.11 to evaluate the expressions, which require 2.0 for unstable
<globin> niksnut: curl https://nixos.org/nixpkgs/packages-unstable.json.gz | gzip -d | jq .commit => cc4677c36ee8d880e881459ad114fd2224b3ac1c
<shlevy> Hmm I wonder if arbitrary curl | jq is safe to execute... :D
<globin> git show cc4677c36ee8d880e881459ad114fd2224b3ac1c => Sun Mar 4 18:31:06 2018
<globin> shlevy: I would NOT do that generally :D
<globin> gchristensen: might be that, yes
<gchristensen> though depends if the channel on the box is ancient
<globin> domenkozar: hmm I've mostly done deploy --force-reboot
<gchristensen> yeah, I've always forced reboots too... we should to some testing of this without a reboot I reckon
<globin> domenkozar: and I haven't been too up-to-date with what's going on with the release
<globin> domenkozar: upgraded all of our infrastructure though with only minor issues, mostly due to stuff being deprecated upstream in dovecot or similar
<domenkozar> hmm :) I had issues with X11
<domenkozar> globin: with nixops?
<globin> domenkozar: yep
pie_ has joined #nixos-dev
<niksnut> why would piping into jq be unsafe?
<fpletz> domenkozar: yeah, that's probably caused by that last minute systemd bump
<fpletz> which in turn fixes some other errors we've been experiencing :/
<domenkozar> NixOS 18.03 To Be Continued released
<shlevy> I'm thinking we should start branch-off the month before :D
<fpletz> ikwildrpepper: domenkozar: do you have any nixops release news?
<domenkozar> afaik it was released?
<fpletz> /o\
<fpletz> thanks
<gchristensen> on Linux + Nix, should `sudo su` then `nix-shell` work?
<gchristensen> I'd love to define some acceptance criteria for this sort of thing, to have a good list of scenarios to test.
<niksnut> globin: should be fixed now
<globin> niksnut: thanks!
xeji has joined #nixos-dev
<clever> gchristensen: sudo -i!
<shlevy> Yeah, sudo -i should work everywhere sane
<gchristensen> I hear that, the trouble is fitting in to what people are used to and not being too annoying. if the acceptance criteria is `sudo -i` works and `sudo nix-shell` doesn't that is fine
coconnor has joined #nixos-dev
<shlevy> At Target policy has root's shell as /bin/false, not to prevent people from accessing a root shell if they have sudo but to prevent logins :|
<shlevy> I'm trying to get that changed but it's the first time I've ever seen that and it definitely seems non-standard/bizarre
<gchristensen> that seems problematic
<shlevy> (this is why the installer does sudo HOME= rather than sudo -i)
Sonarpulse has joined #nixos-dev
<gchristensen> I really like systemd
<ikwildrpepper> fpletz: not yet, wanted to wait to hear if the upgrade issue is a blocker for 18.03
<gchristensen> it is incredible that the Nix Daemon installer for Linux pretty much worked without any special effort on centos7, ubuntu wily, ubuntu xenial, debian jesise, debian stretch, and arch
<niksnut> systemd ftw
<gchristensen> on that topic, anyone have a Linux box that isn't NixOS who would be willing to try out my new installer? :) it requires rm -rf'ing /nix/store
<aminechikhaoui> ikwildrpepper: I thought you did already a release ?
<fpletz> ikwildrpepper: the compat issue was fixed because we didn't expect the 1.6 release before 18.03
obadz has quit [Ping timeout: 240 seconds]
<Sonarpulse> is it a a known issue that the single user nix-profile.sh doesn't handle arbitrary channnels correctly?
<Sonarpulse> it appends nixpkgs=... to NIX_PATH
<Sonarpulse> but doesn't append ~/.nix-defexprt/channels/
<ikwildrpepper> fpletz: the upgrade issue seems to be a NixOS issue
<gchristensen> Sonarpulse: does _any_ system have defexpr in the NIX_PATH?
<Sonarpulse> gchristensen: nixos has that I think
<Sonarpulse> or at least your nix-daemon-profile.sh :D
<gchristensen> mine on nixos is nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs:nixos-config=/etc/nixos/configuration.nix:/nix/var/nix/profiles/per-user/root/channels
<ikwildrpepper> so when upgrading from 17.09 to 18.03, switch-to-configuration exits with non-zero exit code
<gchristensen> the NIX_PATH for the multi-user linux install is just /nix/var/nix/profiles/per-user/root/channels
<ikwildrpepper> (but only produces a warning, and a normal user might not even notice)
<Sonarpulse> gchristensen: err yeah /nix/var/nix/profiles/per-user/root/channels is analogous to ~/.nix-defexpr/channels
<Sonarpulse> nix-channel doesn't really work on non-nixos because of what we have
<gchristensen> Sonarpulse: btw you mention a single-user install. does that mean you'd like to try out •._.••´¯``•.¸¸.•`my new installer`•.¸¸.•´´¯`••._.•
<Sonarpulse> was debugging this with a coworker and sort of embarrissed tbh
<Sonarpulse> gchristensen: hehe can you add option to *not* wipe everything?
<gchristensen> no
<Sonarpulse> I've ran into trouble trying to justify why you've done about that
<Sonarpulse> to others
<Sonarpulse> but maybe I'm justifying it wrong
<gchristensen> I think its just too complicated to get the permissions correct afterwards, and starting with the assumption there isn't one makes the rest of the installer much simpler
<gchristensen> and, IMO, safer. the nix store is easy to repopulate, just takes a bit of time
obadz has joined #nixos-dev
<Sonarpulse> gchristensen: so to be clear i was thinking like an --unsafe mode
<Sonarpulse> so no wiping at the cost of no guarantee of correctness
<Sonarpulse> rather than contorting the installer to try to fix things up
<gchristensen> enterprising installers might look deep inside themselves, find there is no need... then look deep inside the installer, and find there already is one. but I really really don't like the idea of using it.
<Sonarpulse> i sort of think as allowing 2 step process: install the new thing in hacky way and see that it sort of works, then take a deep breath and wipe out old rotton store
<gchristensen> they could mv /nix/store to /old/rotten/store if tehy want
<Sonarpulse> oh
<Sonarpulse> hmm :)
<Sonarpulse> gchristensen: what's the link then?
<Sonarpulse> btw anyone know if the author of https://github.com/timbertson/opam2nix is on IRC?
<gchristensen> let me publish it somewhere good
jtojnar has joined #nixos-dev
<gchristensen> tar -xf that file, ./install
<Sonarpulse> gchristensen: thanks!
<gchristensen> Sonarpulse: https://github.com/grahamc/nix/releases/tag/multi-user-nix-linux-rc1 added instructions here
<shlevy> Sonarpulse: And ignore the $$$miner$$$ process you see in top
<shlevy> That's a known bug
<gchristensen> yeah still trying to fix that little guy
pie_ has quit [Ping timeout: 268 seconds]
<Sonarpulse> gchristensen: ok i think i cannot interrupt work with it now, but I'll give it a shot at the end of the day
pie_ has joined #nixos-dev
<gchristensen> awesome!
<domenkozar> fpletz: what about https://github.com/NixOS/nixpkgs/pull/37988
<domenkozar> I really need to solve my brithday paradox issue with 4M files in /nix/store
<domenkozar> and this allows 3 level deep htree directory index with ext4
<domenkozar> should we have staging-18.03?
<shlevy> we do :)
<domenkozar> shlevy: you mean we do already?
<shlevy> Yep
<shlevy> with a jobset and everything
<domenkozar> I don't see it
<domenkozar> am I going crazy? :)
<gchristensen> well maybe but I don't see it either ;)
<domenkozar> maybe it was deleted?
<shlevy> :|
<shlevy> Thats... odd
<shlevy> Anyway there is a jobset
<shlevy> Let me see if it's disabled !
<shlevy> I have a local checkout of staging-18.03 that references origin/staging-18.03, annoying
<domenkozar> I have pushed the branch with e2fsprogs change
<domenkozar> also added protection to staging-18.03
<ekleog> hmm what's the policy about long-failing modules? I've just tried to setup a gitlab module, but it looks like it fails since jan 2015 https://hydra.nixos.org/job/nixos/trunk-combined/nixos.tests.gitlab.x86_64-linux (and still fails when I try to run it locally). Would it not be better to rm -Rf it (or at least unlink it from the doc) until someone fixes it?
<gchristensen> hard to know if the module is bad or if the test is bad
<gchristensen> given IIRC globin uses that module extensively, I'd bet the test is bad
<LnL> huh, the gitlab module works
<LnL> don't even think I used it before 2016 :p
<gchristensen> broken modules should be deleted yes, but tests don't necessarily mean the module is broken
<ekleog> hmm 'k, well, following the doc from nixos/manual it doesn't work either with the same error message :°
<gchristensen> we should fix that then
<ekleog> (ie. bundler: failed to load command: rake (/nix/store/y9adki02by83s1yingribv1g91pdq15l-gitlab-env-10.5.6/lib/ruby/gems/2.4.0/bin/rake) / Bundler::GemNotFound: Could not find abstract_type-0.0.7 in any of the sources)
<globin> ekleog: ack
<globin> I don't know how to fix apart from reverting the rubygems update
goibhniu has quit [Ping timeout: 248 seconds]
<LnL> oh, so it's broken on 18.03?
* ekleog doesn't know anything about ruby, sorry
<ekleog> yup I'm on 18.03
<globin> that update broke it on 17.09, too
<globin> but I don't have the knowledge and energy to revert or fix ruby infrastructure stuff every time..
<globin> I'll remove it from the module list..
<globin> although I'm pretty sure other rails services will break too
<LnL> it was backported?
<ekleog> maybe just re-adding the gitlab test would make sure this kind of breakage is known at rubygems update time?
ma27 has joined #nixos-dev
<globin> ekleog: the gitlab test has never worked properly
<globin> ekleog: it needs too much IO and times out
<ekleog> oh ok :/ I thought https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/gitlab.nix#L22 was expected to handle that
<globin> ekleog: that worked in the beginning..
<ekleog> oh, so even more than 10min? wow
<globin> ekleog: it works on github.com/mayflower/nixpkgs but we've reverted the rubygems bump..
<gchristensen> ouch
<ekleog> ruby has always been a mess from a package management pov, imo
<ekleog> well, “always”, at least for a few years, didn't use any before
<globin> yep, it needs serious cleaning up, but I have only little knowledge how ruby/gems/bundler work internally and it seems to be quite a mess after having tried to understand parts of it a few times..
<gchristensen> mega ouch
__Sander__ has quit [Quit: Konversation terminated!]
<LnL> globin: have you spent some time on it already or nothing at all?
<globin> LnL: probably about ~10-20 hrs
<LnL> oh ok :/
<globin> and fpletz, too
<domenkozar> is there anything I can help with the release?
<domenkozar> sorry I'm so late in the game :/
<LnL> it's been very long since I did ruby suff so I probably won't get anywhere, but I'll take a look anyway
coconnor has quit [Remote host closed the connection]
MichaelRaskin has joined #nixos-dev
Synthetica has quit [Quit: Connection closed for inactivity]
<fpletz> globin: you mean remove the gitlab module? the frab module is broken, too, and probably others… this bump on 17.09 is unacceptable imho
<fpletz> manveru: zimbatm: what is the reason to bump rubygems? I fixed nothing in upstream nixpkgs, right?
<LnL> I thought it was only master/18.03
<LnL> why was it backported?
<LnL> ah
<fpletz> globin: those issues seem to only be applicable if rubygems is used to do package operations. we don't on runtime but only in the nix sandbox, right?
<fpletz> why not expose rubygems in the fixed version in all-packages but use the old version in our ruby machinery?
<globin> manveru: ^
<Profpatsch> In which output should example configs be put?
<Profpatsch> "data"?
<fpletz> Profpatsch: we don't have any policy for that
<Profpatsch> fpletz: What’s your take?
<Profpatsch> I mean it’s kind of bad that we don’t have a way to consistently browse docs.
<Profpatsch> Lots of packages don’t even package manpages, much less html docs.
<fpletz> yeah, some doc output would be fine but it doesn't make sense to create a new output for only a few kilobytes of doc/data
<Profpatsch> My end goal would be to have a linter that tells you if some files should go into e.g. man, checks that they have no backlinks to out or bin.
<Profpatsch> And then a system to browse global system documentation and stuff.
<Profpatsch> Maybe as a browser thing or in nix-ui
<manveru> fpletz: the reason for rubygems bump was a security issue
<fpletz> yeah, we found the PR
<manveru> that's why i'm not favoring a rollback of that patch
<fpletz> can you say something about my suggestion above?
<manveru> i'll help fixing gitlab or whatever else is broken though
coconnor has joined #nixos-dev
<manveru> i don't think we need the old rubygems version
<manveru> i'm out of the loop though, what exactly is breaking?
<fpletz> iirc some gems like mysql2 can't be found even though they're specified and are in the closure
infinisil has quit [Quit: Configuring ZNC, sorry for the join/quits!]
<fpletz> some months have passed since I tried gitlab with a newer rubygems version
<manveru> yeah, trying to build gitlab on 17.09 atm
<manveru> that's the branch that's failing, right?
<manveru> needs almost 2GB download, so it'll be a while :P
<globin> manveru: using this as vm.nix: https://gist.github.com/
<globin> wow..
infinisil has joined #nixos-dev
<gchristensen> wow, such secure
<globin> %)
<globin> /* DO NOT USE IN PRODUCTION */
<gchristensen> too late
<globin> manveru: I just used that on master to get the gem loading error btw
<globin> reverting my commit disabling gitlab obviously
<zimbatm> fpletz: I think rubygems had a security hole
<zimbatm> oh it was mentioned...
abbradar has quit [Remote host closed the connection]
<manveru> ah, thanks
jtojnar has quit [Quit: jtojnar]
<manveru> globin: gonna try it, don't have much time, but shouldn't be too hard to fix
<thoughtpolice> shlevy: Do you think you or someone else could help me possibly later today (after some late lunch at least!) on uploading something to tarballs.nixos.org? I'm not sure what the protocol is for this...
<thoughtpolice> Or at least what to do or who to ask to do it. :) I'd like to put a jdk10 bootstrap image up there so I can axe JDK 9.
<thoughtpolice> (And we should probably move the other tarballs from dropbox, tbh)
<gchristensen> where do these bootstrap tarballs come from? How do they get built?
<manveru> globin: i'm actually having build failures from the rugged gem
<thoughtpolice> gchristensen: make-bootstrap.nix in pkgs/development/compilers/openjdk, apparently
<globin> manveru: on which branch?
<LnL> gchristensen: magic, and hopefully uploaded to tarballs.nixos.org
acowley has joined #nixos-dev
<shlevy> thoughtpolice: I can upload them for you, yeah
<manveru> nevermind :)
<thoughtpolice> Alright, I'll give a shot to spinning openjdk10 soon then
<manveru> it was old master
<gchristensen> are the bootstraps binary reproducible?
* gchristensen is thinking probably not ;)
<shlevy> Highly doubtful :D
<gchristensen> man those things give me the creeps
<shlevy> Ah, if you prefer we can just use oracle jdk to bootstrap ;)
<LnL> not necessarily, but they are (hopefully) built by a nix expression
orivej has quit [Ping timeout: 245 seconds]
<thoughtpolice> It would be interesting to know if the JDK has made any moves towards reproducible builds; I wonder if Debian has done anything on this...
<shlevy> gchristensen: Did I tell you about my idea to bootstrap a toolchain starting from a dead-simple hex->binary translater? :D
<gchristensen> you have not x.x
<shlevy> Although since Nix comes with a /bin/sh we can do better and start with POSIX sh
taktoa has quit [Remote host closed the connection]
<gchristensen> man I seriously can't believe how easy it was to convert the mac installer to support linux
<dtz> is that what you did?? haha :D
<dtz> how'd you test on various distro's? Is this something we can automate? Would be really great to bang on reliable Nix installer
<dtz> "ideally" on actual kernels and environments they use, to root out things like "nix-shell doesn't work" or "sandboxing doesn't work"
<gchristensen> I wish
<manveru> globin: well, i think i got a fix, you can try it at https://github.com/manveru/nixpkgs/tree/fix-gitlab
<manveru> i gotta go or my wife gets angry :)
<gchristensen> I used Vagrant and just used swapped out different boxes from https://app.vagrantup.com/, dtz
<dtz> oh, okay neat
<gchristensen> it is challenging to automate that nicely in a way that hydra could run it
<dtz> yeah
<gchristensen> a cool thing is the mac portion and the systemd portions are only ~150loc
<MichaelRaskin> Only
<MichaelRaskin> Although if they are that large together, it is not bad
<gchristensen> they are that large individually
* gchristensen sticks to the story that it is cool
<MichaelRaskin> …pleads the fifth on the story that it is cool…
Synthetica has joined #nixos-dev
<zybell> hex translator: try octal: echo abcdefgh|tr abcdefgh '\177\105\114\106\000\001\001\000'>>a.out
<zybell> shlevy:^
davidlt_ has joined #nixos-dev
davidlt has quit [Ping timeout: 252 seconds]
<globin> manveru: doesn't seem to work at all, I get this everywhere: https://gist.github.com/globin/4c0fddcce158a1e1aae85a9ab3ebbc25
<dtz> ./a.out: ELF invalid class LSB (SYSV), unknown class 0
<dtz> :P
<zybell> dtz: example, i had somwhere a shellscript using od to create a sed-script that transformed itself into a shellscript that wrote the bin-file. And it was piped thru sort to get the addr right.
<dtz> xD xD
<zybell> I think you can even create an assembler from sed and bc, now that I think about it.
<MichaelRaskin> Well, sed is turing-complete…
<coconnor> haha
<zybell> ?
<coconnor> @ assember from sed and bc
<coconnor> I bet one could!
<zybell> Nowadays it will be harder, because the machine-lang isn't as well documented as before.
<shlevy> zybell: Obviously we start from RISC-V and cross-compile to x86
<zybell> Cross-compiling with a real gcc? Or sed-assembler?
<manveru> globin: amended the commit, that should be better (at least i don't see restarts in journalctl anymore)
<shlevy> zybell: Real gcc
<shlevy> bootstrap to real gcc with sed-assembler :P
<manveru> globin: basically, don't use `bundle exec`... it's evil :|
<zybell> I think we would need a 'super-simple-C-compiler' as intermediate stage, possibly written in Brainfuck, because I have seen a Brainfuck-Compiler in about 1000 bytes.
<globin> manveru: ok, didn't know about wrappedRuby
<globin> manveru: not sure if that even existed when fpletz and I wrote the gitlab service
<manveru> i don't think it did
<globin> manveru: I'll test that, thanks for this already, I'll report back and try to update gitlab to 10.6 on the way for all versions
<manveru> hope it works out :)
<MichaelRaskin> zybell: and then you don't want to know how many layers of bootstrap you need from a minimal C compiler to GCC 7.0
<zybell> 2 'super-simple-C-compiler' compiles 'gcc-compiling-C-compiler' written in super-simple-C from the days of System6.
<MichaelRaskin> I mean a path with noone ragequitting programming in the process
<MichaelRaskin> Shouldn't be more than ten, actually.
<MichaelRaskin> simple-c → pcc or tinycc → GCC 0.X should hoperfully work, and then it should be possible to go by major versions.
<zybell> 'noone ragequitting'? Unknown expression.
<simpson> MichaelRaskin: If you have two machines, can't LLVM emit simple portable C code and also compile recent GCC?
<MichaelRaskin> I would expect that writing a GCC 7.0 compiling C compiler from scratch in the simple-C language would lead to some of the participants of such an undertaking to leave programming forever, while being in the state of rage.
<MichaelRaskin> simpson: well, GCC could emit simple asm code, would _that_ count?
<zybell> GCC is as of today purposefully written in a C-Dialekt understood by all commercial compilers, old and new.
<simpson> MichaelRaskin: Ha, at that point, I suppose we're just cross-compiling.
<MichaelRaskin> generated C is no better per se
<MichaelRaskin> zybell: I think skipping a majore release of GCC is already painful.
<ekleog> zybell: iirc gcc recently switched to requiring c++, wasn't it?
<sorear> "recently"
<zybell> If you want C++, but C is for the GNU-Project important!
<simpson> Hm. Have we exhausted the topic? I just noticed that this is #nixos-dev.
<zybell> Aren't we developing a process to get nixos on new hw from day one`
<simpson> Sounds like it'd be easier to just make friends with folks who can get you prerelease boards.
<zybell> with an unprecedented level of trust?
<simpson> No, just convince your friend to document the board so that it won't be hard to boot and to send their Linux patches upstream.
<zybell> That line was continuation to preceeding, not answer.
<thoughtpolice> shlevy: I think the jdk bootstrap.nix is going to have to change, because jdk10's "java" command needs zlib in the RPATH :| Which will reimply a rebuild of *all* OpenJDKs, so this is going to hurt a bit...
<simpson> Oh. I dunno. Sounds silly to try to get more trust than what we currently have, given the apparent untrustability of the typical CPU~
<shlevy> thoughtpolice: Fun!
<shlevy> zybell: oh, of course, it's a multi-step process
<shlevy> sed-assembler -> real assembler -> ANSI C -> old gcc -> recent gcc -> new gcc
<thoughtpolice> shlevy: Also I just realized that with the new non-LTS schedule, we'll never be able to remove JDK N-1 before JDK N comes out. :| I would never have even been able to build jdk10 if I had nuked jdk9 beforehand, for example... What a chore.
<zybell> shlevy:sed-assembler -> bf-compiler -> super-simple-c -> gcc-compiling-c -> gcc
<zybell> Only the first step depends on the board.
<shlevy> thoughtpolice: Hmm what do you mean?
<zybell> super-simple-c and gcc-compiling-c can redirect their output through bf-compiler.
<thoughtpolice> shlevy: JDK 10 requires either an existing JDK 10 to bootstrap it, or a JDK 9. I presume this will continue; so for example, once 11 comes out an 10 goes EOL (as it will), we'll still not be able to remove 10 -- until we bootstrap 11 with it.
<thoughtpolice> So we'll always have to hang onto the EOL one until the new one has replaced anything, if necessary. It would be nicer if they had last-LTS-worked policy, but JDK 8 (current LTS) will not work.
<thoughtpolice> It's unclear if a last-LTS-works policy will be true going forward.
<shlevy> We can't bootstrap 11 with the 10 bootstrap?
<thoughtpolice> I suppose so -- I guess *bootstraps* don't matter although keeping EOL versions feels weird.
<shlevy> Yeah, we'll want to make an 11 bootstrap failry quickly too
<thoughtpolice> I plan on bootstrapping 10 to a fixed point with 2 builds: bootstrap 10 via 9, then 10 via 10 and package that as the real one on tarballs.nixos.org
<thoughtpolice> Which I'm about to see if it worked
<shlevy> Yeah, that's how I usally do stdenv and such
<thoughtpolice> Luckily 32 cores means I got done with 3 JDK compiles (and tests!) in less than 15 minutes :)
<shlevy> :D
<dtz> xD
<zybell> thoughtpolice: I think you will need 3 builds: 9 builds 10, 10 builds new 10, new 10 builds 10, compare this 10 with new 10 to prove(!) that you have reached fixed point.
<thoughtpolice> Due to the way the bootstrap works I actually have to go 8(bootstrap) -> 8 (built by bootstrap), 8 -> 9, 9->10, 10->10
<thoughtpolice> But yeah, basically. Although it's not entirely clear if the actual builds are fully deterministic, unfortunately. :( Otherwise ideally just an md5sum across builds would work...
<thoughtpolice> Denied. :( /nix/store/cgcjlszbxbqvss0gbz72qglpb5vz072r-openjdk-bootstrap/lib/openjdk/bin/java: error while loading shared libraries: libfontconfig.so.1: cannot open shared object file: No such file or directory
<zybell> I would assume a 10 build by 9 *would* differ from a 10 build by 10. But a 10 build by 10 shouldn't differ from another 10by10, except for deliberate buildID. cmp -l should help there.
<globin> fyi: I've just removed the ruby dependency that slipped in very deep in the dependency tree, it wasn't actually being used at all: https://github.com/NixOS/nixpkgs/commit/01fb1af4f507a2249668d664575717d84b6c1d89
davidlt_ has quit [Ping timeout: 268 seconds]
xeji has quit [Quit: WeeChat 2.0]
<niksnut> regarding bootstrapping using a chain of ever-more powerful languages, there was a progress report about exactly that at the guix meeting in brussels last month
<niksnut> iirc, they started with something like a 50-byte binary
<niksnut> then built increasingly more powerful scheme variants
<niksnut> before building tinyc, etc.
<dtz> !! awesome!
<niksnut> see also http://bootstrappable.org/
<ekleog> (and then you realise your 50-byte binary is run by 154-kilobytes nix on 30-megabytes Linux, and that both could falsify all it wants, so you're still not free from trusting-trust)
<ekleog> (that said it's still great :D)
<niksnut> running on a cpu with unauditable microcode
<simpson> ekleog: It's a progress report, not a solution report~
<ekleog> and unauditable transistors, too, if we go this way :D (that said it becomes less and less likely imho, as it's harder to distinguish between what you want to infect and normal instructions that would make the backdoor be seen)
<niksnut> I do remember somebody mentioning running in emulators
<pbogdan> globin: is ruby still a build time dependency of webkitgtk?
<MichaelRaskin> ekleog: actually, poisoning transistors might be simpler than microcode
<MichaelRaskin> just because there are more of them
<ekleog> well, transistor and/or microcode both hit the issue that it becomes very hard for them to manage to detect “I'm building gcc so I insert this backdoor”, “I'm building clang so I insert this other backdoor”, etc… “I'm running an AES operation so I do it wrong this way” (oh wait hello AES-NI), while Linux and/or nix/guix could have it much more easily, from my point of view
<ekleog> then you start running your 50-byte compiler from the MBR, and then things go wrong
<ekleog> (that said, 'night :))
<simpson> ekleog: Remember, a CPU backdoor could look like multiplying two 64-bit registers and getting a sometimes-slightly-wrong answer. There's *lots* of ways in which it could do its subtle work.
<simpson> Two 64-bit registers is a 128-bit PSK.
<ekleog> that's what I was thinking about AES operations, but the issue is keeping enough memory to detect you're building a compiler, in this case :)
<MichaelRaskin> ekleog: a good exploit is externally controlled nowadays.
<ekleog> (as I'm considering only trusting-trust attacks, not trusting-the-processor ones)
<ekleog> hmm you think the processor could be remote-controlled when executing certain instructions and that would not be detected?
<MichaelRaskin> ekleog: Intel ME?
<ekleog> well, that was known for a while, the exploit just hadn't been found :p
<MichaelRaskin> Some exploits have
<MichaelRaskin> Intel pretended to patch them
<MichaelRaskin> What I meant is more an exploit that breaks mov when specific 128 bits are seen
<MichaelRaskin> Then an incoming network packet with a very special header gets stashed into Ring -5 or whatever.
<MichaelRaskin> Then it downloads the real payload in the background.
<ekleog> well, for trusting-trust (ie. getting compilers correctly compiled), I think as soon as there's a need for an incoming network packet there isn't much threat, I can just build my compiler on an offline computer
<ekleog> that said an ME-like is a threat, as it could have pre-downloaded the requirements to infect the compiler while it's being build (though I'd rather go like ring -3 for that)
<ekleog> but doing it just from a clever manipulation of the transistors… I'm still skeptical, it's too risky if it ever gets detected, and there is no plausible deniability :)
<MichaelRaskin> Have you read the papers on the topic?
<ekleog> a few, likely not all
<MichaelRaskin> The point is that it can be done as a relatively minor manufacturing defect, so most methods of layout control would not distinguished the result from a non-backdoored CPU.
<ekleog> the best one for me was the one about aes-ni and the fact that if you changed slightly the operating conditions of the processor it starts to insert errors and allow the key to be recovered, iirc :)
<MichaelRaskin> Oh true.
<thoughtpolice> oh my god -- Potential Boot JDK found at /nix/store/qk32y17xx10vd931mkrz1q6yv2dh5xm3-openjdk-bootstrap/lib/openjdk is incorrect JDK version (/nix/store/qk32y17xx10vd931mkrz1q6yv2dh5xm3-openjdk-bootstrap/lib/openjdk/bin/java: error while loading shared libraries: libXinerama.so.1: cannot open shared object file: No such file or directory
<thoughtpolice> There has to be a better way than passing every single v10 dependency through the bootstrap script...