gchristensen changed the topic of #nixos-dev to: NixOS Development (#nixos for questions) | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html | 18.09 release managers: vcunat and samueldr | https://logs.nix.samueldr.com/nixos-dev
lassulus has quit [Ping timeout: 250 seconds]
q6AA4FD has quit [Quit: ZNC 1.7.1 - https://znc.in]
copumpkin has quit [Ping timeout: 244 seconds]
lassulus has joined #nixos-dev
drakonis has quit [Quit: WeeChat 2.3]
lassulus has quit [Ping timeout: 250 seconds]
lassulus has joined #nixos-dev
Synthetica has quit [Quit: Connection closed for inactivity]
lassulus_ has joined #nixos-dev
lassulus has quit [Ping timeout: 240 seconds]
lassulus_ is now known as lassulus
pie___ has joined #nixos-dev
pie__ has quit [Ping timeout: 240 seconds]
init_6 has joined #nixos-dev
q6AA4FD has joined #nixos-dev
init_6 has quit [Ping timeout: 250 seconds]
init_6 has joined #nixos-dev
lassulus has quit [Ping timeout: 240 seconds]
lassulus has joined #nixos-dev
init_6 has quit [Ping timeout: 250 seconds]
lassulus has quit [Ping timeout: 268 seconds]
lassulus_ has joined #nixos-dev
lassulus_ is now known as lassulus
<Profpatsch> Everybody who is at FOSDEM, come join #nixos-fosdem!
obadz has quit [Quit: WeeChat 2.3]
obadz has joined #nixos-dev
averell has quit [Ping timeout: 252 seconds]
averell has joined #nixos-dev
orivej has quit [Ping timeout: 240 seconds]
delroth has quit [Ping timeout: 250 seconds]
tilpner has joined #nixos-dev
delroth has joined #nixos-dev
<gchristensen> reading the queue runner is cool, because it confirms so much behavior I have only up until now intuited
<gchristensen> hydra's build queue could be a graph database with a fancy webpage showing where in the graph builders are building
<clever> gchristensen: there is also a magic url to dump the current status, and ive also written some utils to directly query the db for perf metrics
<clever> yeah, thats the one
<{^_^}> input-output-hk/iohk-ops#409 (by cleverca22, 22 weeks ago, open): [DEVOPS-1012] adding first version of hydra-util to git
<clever> gchristensen: this PR adds a util that will expose some timing details from the hydra db
<gchristensen> nice
<MichaelRaskin> gchristensen: Hydra could also provide a single-request export of public jobset jobs across time…
<clever> gchristensen: basically, it allows you to render a graph, showing the timespan every step was building for
<clever> and then you can see if a group of steps are waiting for one step
<MichaelRaskin> Actually, with graph it might be better to allow dumping and processing locally
<clever> or if there is a gap, where nothing built at all (slaves busy with other things)
<MichaelRaskin> Although if it is JS, a webpage with scripts to process locally will arise
<clever> it also deals with other jobsets stealing builds and caching them
<gchristensen> huh
<clever> if you query for every step in the current eval, you wont see cached builds done by another jobset, on the same drv
<clever> but the above util will query the closure of everything in the eval, then find the steps that built it, no mater what eval/jobset built it
orivej has joined #nixos-dev
<gchristensen> I'm a bit suspicious that hydra doesn't properly utilize hardware across architectures / features
delroth has quit [Quit: WeeChat 2.3]
delroth has joined #nixos-dev
<gchristensen> to clarify, nixpkgs:nixpkgs-unstable might have a zillion aarch64 build jobs, but be deprioritized b/c it is out of shares, meanwhile no other jobset has aarch64 jobs
<MichaelRaskin> I dunno, is hydra.nixos.org the web frontend underloaded enough for scraping it being a better solution than negotiating a data dump?
<MichaelRaskin> (of course, that won't give an answer to questions like «does a nonpublic jobset compete»)
<gchristensen> there are no non public jobsets
<gchristensen> unless even admin rights can hide them ...
<MichaelRaskin> Wait, then _what_ are the privacy implications of a Hydra data dump?
<clever> private jobsets are a joke in hydra
<clever> MichaelRaskin: i believe the sha1 of passwords is in there, if using hydra based login
<gchristensen> user accounts
<MichaelRaskin> I mean, come one, not exporting a table is not that complicated.
<gchristensen> pass me the command :)
<gchristensen> preferrably the dump will go to stdout, not the fs
<MichaelRaskin> That's the default
<MichaelRaskin> pg_dump -d DB_NAME -T TABLE_TO_SKIP
<MichaelRaskin> Skipping a _column_ might be harder
<MichaelRaskin> I think user table doesn't affect the queue in any way anyway
<clever> i believe the queue-runner will mostly be checking the closure of a given .drv (nix-daemon intensive), then checking what paths exist already (s3 intensive), then computing what steps remain to build, and adding them to an internal queue
<gchristensen> this look right? sudo -u hydra pg_dump -T users -T userroles -T starredjobs -T projectmembers | pv | gzip | aespipe | ssh remote-server 'cat > ./exports/hydra-dump-2019-02-02.gz.aespipe'
<clever> looks good at a glance
<clever> you can also reverse the ssh a bit, ssh hydra@hydra -t "pg_dump ... | pv | gzip | aespipe"
<gchristensen> heh, true
<clever> or ssh anything@hydra -t "sudo -u hydra ..."
<gchristensen> that is very complicated
<clever> -t forces it to allocate a tty, so sudo and friends will work as normal
orivej has quit [Ping timeout: 268 seconds]
<MichaelRaskin> Why aespipe if it is supposed to be a dump of public data?
<simpson> It's good to be in the habit. And that's the right order, too.
<gchristensen> and, just in case I accidentally included non-public data
<simpson> Well, um. `pv` should be later in the pipe. You probably don't care how fast Pg goes, but how fast Pg and gzip together go. Unless this is one of your forty-something-core boxen, I guess?
<clever> gchristensen: oh, you can make the gzip faster...
<dtz> maybe let ssh handle compression/encryption (setting those options explicitly if needed)?
<clever> pigz.out 126,848 x /nix/store/hx121lsjmk768nv5x928bhpn2xyz1n2k-pigz-2.3.3/bin/pigz
<clever> pigz, will gzip in parallel
<dtz> but that's not really important and this way it stays that way :)
<clever> `gzip -9` will take 900kb blocks of input, and compress each, in series
<dtz> *stays encrypted, never leaves host unenc by accident, etc.
<simpson> dtz: ssh can't not encrypt. The idea is that the blob will be encrypted *even if* the other end is controlled by Eve or Mallory.
<clever> pigz, just takes each 900kb block, and compresses it in a different thread, and then re-assembles the compressed blocks
<gchristensen> simpson: I do want to know what the write end is doing
<gchristensen> though, really, I'd like a pv at the start and end of that chain
<simpson> dtz: And then application of Cryptographic Principle of Doom forces us to compress before encrypting, defeating any compression ssh might apply. (ISTR that SSHv2 compression is risky anyway?)
<simpson> gchristensen: I think that you can do that! TIAS with another pipe first.
<gchristensen> yeah, I could have
<gchristensen> also, compressing prior to encryption exposes some attacks (CRIME)
<simpson> I don't know if there's CRIME for SSHv2 out there, but the principle exists, and I found a mailing-list post where somebody who is not you is asking about whether CRIME could affect their use of OpenSSH to transfer Pg backups.
<gchristensen> interesting :)
orivej has joined #nixos-dev
avn has quit [Ping timeout: 246 seconds]
avn has joined #nixos-dev
orivej has quit [Ping timeout: 240 seconds]
<samueldr> (additionally, not sure the e-mail from aszlig to the linux-unioinfs list made it :/)
ckauhaus has quit [Quit: WeeChat 2.2]
<dtz> :(
<samueldr> ouch
<samueldr> even though they maybe fixed a bug (honouring this constant) this is breaking the userspace
<dtz> that's my understanding, yeah. And even if they bump the default, or we bump ours... it's uncomfortable to rely on folks' kernels not enforcing the limit it's had for ages lol :/. idk hopefully there's a good resolution ... O:)
<samueldr> aszlig: sorry to ping you again, that's just in cas your client is sensitive to beginning of a line; I'm really not a mailing list person, so I wouldn't know what to do about the presumably lost e-mail
<aszlig> samueldr: well, i could bounce it again, but not sure that would change anything
<aszlig> samueldr: another way would be to ping the maintainer via github...
<samueldr> maybe reply without CCing anyone nor the list to the original recipient, without the long e-mail, asking if they received it?
<samueldr> I'm thinking maybe the CCs+attachments are the root cause?
<aszlig> samueldr: i did that already
<samueldr> ah
<aszlig> but no reply
<samueldr> :/
<aszlig> ... on january 31
<samueldr> thanks aszlig <3
__Sander__ has joined #nixos-dev
<aszlig> samueldr: well, the CCs shouldn't be a problem... but maybe the attachments were?
<samueldr> anything is plausible
<aszlig> samueldr: i'll send again without the attachments
<aszlig> 5A560980DD95A: to=<linux-unionfs@vger.kernel.org>, relay=vger.kernel.org[]:25, delay=4, delays=0.36/0.05/1.1/2.5, dsn=2.7.0, status=sent (250 2.7.0 nothing apparently wrong in the message. BF:<H 0.000907487>; S1726506AbfBBR3a)
<aszlig> samueldr: now it worked
<samueldr> I just realized the android app for gmail shows you as an octogonal interrogation mark, which means "could be a spammer"; the description is "Unauthenticated email; Gmail couldn't verify that nix.build actually sent this message (and not a spammer)." which sounds like something that with other criterions would count against you :/
<aszlig> samueldr: yah, that's probably because i'm missing an SPF record
<samueldr> yeah
<aszlig> which shouldn't be an issue with the ML
__Sander__ has quit [Quit: Konversation terminated!]
<aszlig> i mean, when looking through the archives i found real spam with forged sender addresses, which went through
<aszlig> samueldr: i'll post the mailing list link as soon as it's on spinics.net, because marc.info pretty much garbles up the mail =)
<samueldr> thanks again <3
<samueldr> once the ball is rolling with a dicussion there I hope some progress is going to happen
drakonis1 has joined #nixos-dev
<gchristensen> ooooooooooh crap
<samueldr> uh?
<gchristensen> the shebang problem :D
<samueldr> right
* samueldr is swimming in kernel regressions
<samueldr> gchristensen: what's your opinion for the new_kernel medias (iso image, sd image) and zfs? what should be done?
* samueldr gasps in java
<samueldr> in job ‘nixpkgs.chromium.aarch64-linux’: You must accept the Oracle Binary Code License Agreement for Java SE at ...
<gchristensen> imo? leave it failing until 0.7.13 is out
<samueldr> gchristensen: I figure it will continue failing in the future
<gchristensen> oh?
<samueldr> new_kernel is a moving target that will not be updating in lock-step with zf
<samueldr> zfs*
<gchristensen> sure..
<gchristensen> what is this new_kernel thing?
<gchristensen> and, what is its promise / does it get published anywhere?
<samueldr> uses linuxPackages_latest instead of the default (which is usually latest LTS)
<samueldr> not published except on hydra, but its usefulness is in two parts
<aszlig> nixpkgs.chromium.aarch64-linux - wait a minute, chromium requires oracle java? O_o
<samueldr> (1) installing on new or fussy hardware; a couple of times I was able to help a new user installing
<samueldr> aszlig: possibly "any" java
<aszlig> samueldr: why?
<samueldr> aszlig: but I think the java situation on aarch64 isn't well figured out (might be wrong)
<samueldr> aszlig: why? I don't know yet
<samueldr> might be a build-time dep
<samueldr> it's on 18.09, haven't verified yet on unstable whether the same applied
<samueldr> (I think there was progress on the java front in unstable)
<samueldr> (2) for some aarch64 use, new_kernel is probably the most useful variant of the sd_image, especially when figuring out new~ish hardware
<aszlig> wtf O_o
<samueldr> but (1) alone imho is really useful
<gchristensen> well, my preference would be to keep zfs in, and let it fail when it fails.
* aszlig is glad he's no longer using chromium.
<gchristensen> but that is just my preference
<samueldr> gchristensen: since the addition of zfs to the installer, there was (IIRC) no new_kernel image built :/
<samueldr> I would gladly agree if the issue was transient and happened on bumps, though
<samueldr> and maybe it's an unlucky streak, too
<gchristensen> I think it is transient due to more severe than usual breakage
<gchristensen> but it is just my preference
<aszlig> okay, turns out it's a build-time requirement for the mojo bindings: https://chromium.googlesource.com/chromium/src/+/72.0.3626.28/ui/webui/webui_features.gni
<gchristensen> anyone have a feel for how the kernel team sees NixOS?
<samueldr> I only reported one issue to the kernel, but haven't felt *anything*, for my report it looks like it was an implementation detail to the issue (the issue was provably not nixos specific)
<gchristensen> hrm
<gchristensen> ok
<aszlig> first need to have a vm test that actually reproduces the problem...
<MichaelRaskin> gchristensen: what do you want to report that kernel developers' opinion of NixOS is omportant?
<gchristensen> I don't understand the question
<MichaelRaskin> > <gchristensen> anyone have a feel for how the kernel team sees NixOS?
<{^_^}> error: syntax error, unexpected ')', expecting ID or OR_KW or DOLLAR_CURLY or '"', at (string):218:1
<gchristensen> I don't care too much about what they think, but am aware that the context might matter. "breaking userspace for debian" is a different scale than "breaking userspace for Hannah Montana Linux".
<MichaelRaskin> Well, they also probably break some amount of userspace pip installations if they strengthen the check
<gchristensen> yeah
<MichaelRaskin> For Nixpkgs kernels we might start to bump the limit already.
<gchristensen> and anyway, just curious
<gchristensen> it is cool to see (for example) the broad change in opinion of Debian team members of Nix :)
<MichaelRaskin> Well, I guess we did demonstrate the ability to survive long enough to be worth _discussing_ an exception seriously
<gchristensen> yeah :) definitely
<MichaelRaskin> The argument «FHS is to prevent surprises, if you install Nix explicitly, you will be _surprised_ not to see /nix» is actually a reasonable one
<gchristensen> we should have a 16-year party on june 15
<ekleog> 2**2**2-year party
<ekleog> next one will be in 67539, so it's a once-in-a-lifetime chance
worldofpeace has joined #nixos-dev
<gchristensen> :D
<infinisil> > pow 3 (pow 3 3)
<{^_^}> 7625597484987
<infinisil> > pow 2 (pow 2 (pow 2 2))
<{^_^}> 65536
<infinisil> Ahh
<aszlig> ekleog: or if you use bits, it's in 2035 which would at least be twice-in-a-liftime :-)
<aszlig> or for some people even three times =)
JosW has joined #nixos-dev
q6AA4FD has quit [Ping timeout: 246 seconds]
domenkozar has quit [Ping timeout: 250 seconds]
q6AA4FD has joined #nixos-dev
JosW has quit [Ping timeout: 252 seconds]
JosW has joined #nixos-dev
Cale has quit [Ping timeout: 250 seconds]
Cale has joined #nixos-dev
JosW has quit [Quit: KVIrc 4.2.0 Equilibrium http://www.kvirc.net/]
<infinisil> Question: For a NixOS option that controls whether the configuration is declarative via NixOS or stateful, what should it be named?
<infinisil> Should I have a `stateful` option, or a `declarative` one?
<infinisil> Existing options of similar purpose: `services.znapzend.pure` and `security.dhparams.stateful`
<worldofpeace> Declarative operation should be expected, so I'd go with `stateful`.
<infinisil> One thing I should mention is that I'll have to make it stateful by default for backwards compatibilities sake
<worldofpeace> Though better names are always better :D
<infinisil> worldofpeace: You still think it should be `stateful` with a default = true?
<worldofpeace> infinisil: I think so
<infinisil> Because I almost feel like `declarative` with a default of false would be better, since then the user needs to set a positive value if they want it declaratively (which is something positive)
<ekleog> infinisil: there's also `services.openldap.declarativeContents`
<ekleog> I personally like `declarative`
<infinisil> Me too right now
<ekleog> as for the backwards compatibility matter… well, we *can* break backwards compat between releases if we want to, can't we? whether it's a good idea is another question
<worldofpeace> A release should suffice then, no?
<infinisil> I personally am annoyed every time when something breaks, I'd rather not bring this on people
<ekleog> well, I'm too, but I run release-18.09 for this reason, and read release notes
<ekleog> (and am definitely annoyed when a release note is missing, but that's another question :p)
<ekleog> ideally there'd be an intermediate release of only warning, though
<infinisil> Okay so the concrete usecase is minecraft-server. Currently you can only run it statefully in NixOS, but there's a PR to add declarative options
<worldofpeace> Now I get it :D
<infinisil> And the config file of minecraft always sets all options, even when it could just use the defaults, so it would be a bit of a pain to have to figure out how to translate that to a NixOS option
<infinisil> Because you don't want to explicitly set the defaults, but also you're not really sure what the defaults are
<ekleog> why wouldn't you want to explicitly set the defaults? they're set in the stateful config file anyway, IIUC
<infinisil> Yeah, but you don't need them, I don't want to set all 25 values in my configuration.nix when all of them but one are the defaults anyways
<ekleog> well, you don't need them, but I mean it wouldn't change any visible behavior if you did (and actually not setting them would change visible behavior after an upgrade of the server that changes the defaults)
<infinisil> And anyways, there is also a whilelist option, which needs playername and playeruuid, which can be annoying, since I believe minecraft has commands to add to that file ingame
<ekleog> that… sounds like a PITA
<ekleog> no way to have those split to a separate config file I guess?
<infinisil> By 'those' you mean?
<ekleog> the whitelist's playername&uuid
<infinisil> Oh, whitelist is a different file
<ekleog> hmm'k
<ekleog> so it means having some script for updating the whitelist like for /etc/passwd / groups, I guess?
<infinisil> Eh, I think it's fine to be stateful by default
<infinisil> There's no harm to this
orivej has joined #nixos-dev
<ekleog> indeed, sounds like it's hard-to-impossible to not be stateful anyway
<ekleog> like the nextcloud config and issues that ensue :/
<infinisil> If you don't care about minecraft changing your configs then it shouldn't be a problem
<infinisil> (because it can't if its declarative)
<infinisil> Alright I think I'll name the option `declarative`
<infinisil> I'll also need a `.declarative` file in the dataDir to detect state/decl, so this fits
drakonis1 has quit [Quit: WeeChat 2.3]
orivej has quit [Remote host closed the connection]
orivej has joined #nixos-dev
orivej has quit [Remote host closed the connection]
orivej has joined #nixos-dev
orivej has quit [Read error: Connection reset by peer]