worldofpeace_ changed the topic of #nixos-dev to: #nixos-dev NixOS Development (#nixos for questions) | NixOS 20.03 BETA Announced https://discourse.nixos.org/t/nixos-20-03-beta/5935 | https://hydra.nixos.org/jobset/nixos/trunk-combined https://channels.nix.gsc.io/graph.html | https://r13y.com | 19.09 RMs: disasm, sphalerite; 20.03: worldofpeace, disasm | https://logs.nix.samueldr.com/nixos-dev
<{^_^}> firing: RootPartitionLowDiskSpace: https://status.nixos.org/prometheus/alerts
<gchristensen> oh hehe the macs are all starting to fire
<cole-h> I hope this is figurative fire this time... 👀
<gchristensen> fingers crossed
<gchristensen> though if they're on fire themselves, it will have to be ikwildrpepper who spills blood on them
abathur has quit [Ping timeout: 265 seconds]
<gchristensen> shoulda done this sync before my system died https://status.nixos.org/grafana/d/5LANB9pZk/per-instance-metrics?orgId=1&refresh=30s&var-instance=haumea:9100&from=1584720900000&to=now&fullscreen&panelId=13
<andi-> whats that transfer?
<gchristensen> zfs snapshots of the postgres database
<gchristensen> in case my pool is toast
<{^_^}> firing: RootPartitionLowDiskSpace: https://status.nixos.org/prometheus/alerts
Emantor has quit [Quit: ZNC - http://znc.in]
Emantor has joined #nixos-dev
<{^_^}> resolved: RootPartitionLowDiskSpace: https://status.nixos.org/prometheus/alerts
drakonis has quit [Quit: WeeChat 2.7.1]
<julm> gchristensen: out of curiosity, what tool or commands are you using for this zfs backup?
<gchristensen> znapzend
<julm> gchristensen: appreciated, thanks!
<gchristensen> you're welcome!
abathur has joined #nixos-dev
abathur has quit [Ping timeout: 240 seconds]
drakonis has joined #nixos-dev
<julm> last month I spent a fez hours to start reviewing the alternatives for ZFS backuping, it's quite a lot: zfs-auto-snapshot (Bash), zfs_autobackup (Python), zfsnap (sh), znapzend (Perl), zrepl (Go), and of course custom zfs send/receive (+ mbuffer optionally). I still have to motivate myself to resume this reviewing and choose something to try
<gchristensen> and syncoid
<julm> I've just checked something, looks like calling pg_{start,stop}_backup() is indeed not necessary with ZFS snapshoting: https://www.postgresql.org/message-id/8B1637E3-3FC8-450A-9E7C-5035417C2C48%40likeness.com
<gchristensen> correct :)
<gchristensen> as long as your entire postgresql state and data and tablespace directories are in the same pool, and they're atomically snapshotted together
evils has quit [Ping timeout: 256 seconds]
<gchristensen> if you recover from a snapshot, it will start as if it crashed
<julm> and syncoid (Perl), pfiou.. :s
evils has joined #nixos-dev
<julm> also, I've read it's good practice to have one dataset per database
<gchristensen> probably depends on your workload, but couldn't hurt as long as they are all in the same pool
<gchristensen> (otherwise recursive snapshops aren't atomic)
<gchristensen> cool
<julm> slide 101
<julm> thanks!
<julm> hehe, that pdf is already there
<gchristensen> :)
<gchristensen> I did a lot of research before doing it, I didn't want to move hydra and have it be significantly slow because it was a bad idea in the first place
<julm> and so, is it performing as well as wanted?
<gchristensen> it does pretty well
<gchristensen> nobody here knows hydra.nixos.oorg to be fast, so there was some wiggle room :P
<julm> great :D
orivej has quit [Ping timeout: 250 seconds]
abathur has joined #nixos-dev
abathur has quit [Ping timeout: 250 seconds]
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
zarel_ has joined #nixos-dev
zarel has quit [Ping timeout: 240 seconds]
cole-h has quit [Quit: WeeChat 2.7.1]
phreedom has quit [Ping timeout: 240 seconds]
phreedom has joined #nixos-dev
avn has quit [Ping timeout: 264 seconds]
phreedom has quit [Read error: Connection reset by peer]
phreedom has joined #nixos-dev
abathur has joined #nixos-dev
abathur has quit [Ping timeout: 240 seconds]
MichaelRaskin has joined #nixos-dev
MichaelRaskin has quit [Quit: MichaelRaskin]
MichaelRaskin has joined #nixos-dev
Jackneill has joined #nixos-dev
Jackneill has quit [Ping timeout: 240 seconds]
MichaelRaskin has quit [Ping timeout: 246 seconds]
abathur has joined #nixos-dev
abathur has quit [Ping timeout: 246 seconds]
__monty__ has joined #nixos-dev
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
sogatori has joined #nixos-dev
orivej has joined #nixos-dev
teto has joined #nixos-dev
avn has joined #nixos-dev
abathur has joined #nixos-dev
Jackneill has joined #nixos-dev
abathur has quit [Ping timeout: 264 seconds]
avn has quit [Quit: Lost terminal]
avn has joined #nixos-dev
greizgh has quit [Quit: greizgh]
greizgh has joined #nixos-dev
orivej has quit [Ping timeout: 264 seconds]
abathur has joined #nixos-dev
orivej has joined #nixos-dev
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
CRTified has quit [Ping timeout: 250 seconds]
bhipple has joined #nixos-dev
<jtojnar> ryantm did the gnome blacklist stop working?
<Ericson2314> niksnut: I think we can have unprivilaged drv-file-less building without changing the hashing scheme!
<Ericson2314> The trick is, the hashes the hashModuloDerivation puts in don't actually matter
<Ericson2314> (hashes and outputs)
<Ericson2314> it's just arbitrary data whose only purpose is to compute the store path
<gchristensen> I hope there are no plans to eliminate drv files altogether, I use them quite significantly
<Ericson2314> gchristensen: oh no absolutely not; in fact I want more of them!
<gchristensen> uhoh
<Ericson2314> :)
<gchristensen> :)
<Ericson2314> The issue today is that buildPath require sending over extra data, but is unprivilaged
<Ericson2314> while buildDerivation allows sending over just what is needed, but is privilaged
<Ericson2314> but we can have the past of both worlds, by sending over basically the thing that hashModuloDerivation hashes
<Ericson2314> which can be thought of as BasicDerivation + salt
<Ericson2314> there's nothing privilaged about choosing the salt (we already prevent primage attacks that would come from that being wrong)
<gchristensen> what about reuse?
<Ericson2314> gchristensen: if the client makes the "honest" choice of the salt, then they get reuse
<Ericson2314> if they spoof it, then they don't
<gchristensen> I mean salt reuse
<Ericson2314> oh I shouldn't call it "salt" then
<gchristensen> usually you want a good salt, and what happens if we can't trust it to be good?
<gchristensen> ah
<gchristensen> I need to go outside and sterilize some boxes before I bring them inside. back in a few :)
<Ericson2314> so right now hashModuloDerivation puts some hashes of dependend-upon derivatitons in there
<Ericson2314> it doesn't do anything except influence the store path
<gchristensen> that store path is pretty important though, can we validate it?
<Ericson2314> gchristensen: OK, enjoy everyone being 6 >>> feet away in the berkshires!
<gchristensen> :P thanks!
<Ericson2314> gchristensen: we validate it up to these hashes, but that's it
<Ericson2314> nothing more is ever needed
<gchristensen> Ericson2314: so are you saying what was being done is totally unneeded, or that the client will be able to lie in some innocuous way? being able to influence the store path seems shady
<{^_^}> firing: RootPartitionLowDiskSpace: https://status.nixos.org/prometheus/alerts
Jackneill has quit [Ping timeout: 256 seconds]
<Ericson2314> gchristensen: sorry, some lag in Riot. The client will be able to lie in an innocuous way, but it's OK. It was always redundant to hash *how the inputs are built* when all that is required is knowing what the inputs are, for computing store paths
<Ericson2314> you can "make up whatever story you want" i.e. chose weird hashes, for how the inputs are made, but it doesn't matter as you aren't actuallly deciding how they are made, and also require that the remote builder has already built them
<Ericson2314> it's like the client is comming up with some alternate history which may or may not be correct, and the remote builder doesn't care
<Ericson2314> Already, there is no way to know which store path corresponds to which input derivation
<Ericson2314> except for having all the DRVs to redo all the hashing
Jackneill has joined #nixos-dev
justanotheruser has quit [Ping timeout: 256 seconds]
drakonis_ has joined #nixos-dev
drakonis has quit [Ping timeout: 250 seconds]
drakonis_ has quit [Read error: Connection reset by peer]
drakonis has joined #nixos-dev
drakonis_ has joined #nixos-dev
justanotheruser has joined #nixos-dev
drakonis has quit [Ping timeout: 246 seconds]
cole-h has joined #nixos-dev
teto has quit [Ping timeout: 246 seconds]
teto has joined #nixos-dev
drakonis has joined #nixos-dev
drakonis_ has quit [Ping timeout: 246 seconds]
drakonis1 has joined #nixos-dev
drakonis_ has joined #nixos-dev
drakonis has quit [Ping timeout: 246 seconds]
teto has quit [Ping timeout: 246 seconds]
drakonis_ has quit [Ping timeout: 240 seconds]
teto has joined #nixos-dev
drakonis_ has joined #nixos-dev
<bgamari> gchristensen, have there been any further developments on #69360?
<{^_^}> https://github.com/NixOS/nixpkgs/issues/69360 (by arianvp, 25 weeks ago, closed): Bonded network not working on Packet.net
<bgamari> gchristensen, the workaround suggested in late November does not appear to work for me
<domenkozar[m]> srk: spring works :D
<domenkozar[m]> srk: let me know if you want to play
teto has quit [Ping timeout: 246 seconds]
<srk> domenkozar[m]: pretty wasted but can test for sure
<srk> need to learn controls :)
teto has joined #nixos-dev
Guest29 has joined #nixos-dev
Guest29 has quit [Client Quit]
tsaeger has joined #nixos-dev
tsaeger has quit [Quit: Textual IRC Client: www.textualapp.com]
tsaeger has joined #nixos-dev
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
<domenkozar[m]> srk: sorry got into a game and didnt know how to get out :D
<srk> lol :D
<srk> is it that catchy?
<{^_^}> firing: RootPartitionLowDiskSpace: https://status.nixos.org/prometheus/alerts
<domenkozar[m]> srk: it's pretty good
<arianvp> Darnit bgamari
<bgamari> oh?
<arianvp> I suppose you found out when updating ghc gitlab? :P
<bgamari> yep
<arianvp> (that the workaround doesn't work)
<bgamari> and unfortunately the old system configuration unexpectedly vanished from the bootloader menu
<bgamari> so now I'm trying to debug on the fly
<bgamari> I now know how to bring the bond interface up manually
<bgamari> I just can't figure out for the life of me how to get networkd or nix's bonding script to do so
<bgamari> frankly I don't see how that workaround could have ever worked
<bgamari> the bonding script itself makes no attempt to set the MAC address
<bgamari> and it seems that the interface script doesn't run until after the bonding interface is up
<arianvp> I need to pick my 4 month old brain
<arianvp> Maybe we had networkd networking enabled instead of scripted?
<bgamari> perhaps
<bgamari> but it would have required some effort
<bgamari> unfortunately my experience thusfar shows that networkd also fails to set the mac address
<bgamari> does anyone know how to get `systemctl start` to show you what it's doing?
<bgamari> instead of just hanging
<bgamari> I've brought up the bonding interface manually and confirmed that I have connectivity
<bgamari> but `systemctl start default.target` inexplicably hangs
<bgamari> yet `systemctl list-units --failed` claims there are no failed units
<{^_^}> firing: RootPartitionLowDiskSpace: https://status.nixos.org/prometheus/alerts
<flokli> bgamari: if you're still debugging packet.net bonds, try describing things via the systemd.network.links (it now works in unstable and 20.03) - https://github.com/NixOS/nixpkgs/pull/82941
<{^_^}> #82941 (by flokli, 2 days ago, merged): nixos/systemd: apply .link even when networkd is disabled (without the lib refactor this time)
<flokli> I'm not sure if the bash script thing works reliably - but this is now directly in udev
<arianvp> I think he's still on 19.09
<arianvp> Or .03
<flokli> well, then not ;-)
<bgamari> flokli, I am
<flokli> bgamari: you are what?
<bgamari> still debugging, that is
<flokli> ok
<flokli> so, you might want to try configuring the mac address via systemd.network.link
<bgamari> flokli, however, I'm already using master
<bgamari> flokli, I am now doing so
<bgamari> unfortunately networkd seems not to apply the change
* bgamari is using 19fc3f3230ea1a31996dbc887603dee683cd39bb
<flokli> bgamari: networkd doesn't apply the .link stuff
<flokli> that's udev
<flokli> but… before that PR, things were only applied IF you also enabled networkd
<flokli> which is wrong
<bgamari> FWIW I have also enabled networkd at this point
<flokli> so, if you're nixpkgs is past that merge or the backport to 20.03, you can try with the .link options
<flokli> well, then what /etc/systemd/network/*.link files do you have?
<flokli> and what does networkctl status $bondIf say which .link file it takes?
drakonis1 is now known as drakonis
<bgamari> flokli, /etc/systemd/network/bond0.network contains Link.MACAddress
<bgamari> but indeed perhaps it needs to be in a .link file
<bgamari> I'll give that a try
<flokli> bgamari: yes, needs to be .link ;-)
<flokli> but seriously, this is all just working around the problem, and packet.net needs to fix their switch config.
<flokli> with other distros, people will run into this as well…
ixxie has joined #nixos-dev
* domenkozar[m] notes to avoid packet for a little while longer
<bgamari> flokli, sure
<bgamari> flokli, I'm just in the unfortunate position of having to make it work until they decide to do so
<bgamari> I've opened a ticket, FWIW
<flokli> bgamari: very good. Point them to the NixOS PR, too :-)
<flokli> s/PR/issue/
sogatori has quit [Remote host closed the connection]
<{^_^}> resolved: RootPartitionLowDiskSpace: https://status.nixos.org/prometheus/alerts
lassulus has quit [Ping timeout: 272 seconds]
lassulus has joined #nixos-dev
<bgamari> flokli, unfortunately even with the MACAddress definition in bond0.link it still isn't applied
* bgamari has no idea where the MAC address that *is* applied came from
<bgamari> it's shared between both physical interfaces and the bonding interface
<bgamari> flokli, do you know how one might ask udevd whether it sees my .link file?
<bgamari> `udevadm info /sys/devices/virtual/net/bond0` makes no mention of the MAC address I provided
<bgamari> but it's unclear why
<flokli> bgamari: networkctl status $interfacename
<flokli> should show which file udev pick(ed)
<flokli> might need a reboot
<bgamari> thanks!
<flokli> np
<bgamari> sigh, apparently this doesn't work in the emergency console
<bgamari> Failed to connect system bus: No such file or directory
<{^_^}> firing: RootPartitionLowDiskSpace: https://status.nixos.org/prometheus/alerts
justanotheruser has quit [Ping timeout: 250 seconds]
orivej has quit [Ping timeout: 240 seconds]
teto has quit [Ping timeout: 260 seconds]
teto has joined #nixos-dev
justanotheruser has joined #nixos-dev
orivej has joined #nixos-dev
m1cr0m4n has joined #nixos-dev
<bgamari> gchristensen, Does the 19.03 Packet image work at all?
<bgamari> gchristensen, I brought up a fresh c1.small.x86 on EWR and found that it too comes up without any network connectivity
m1cr0m4n is now known as m1cr0man
teto has quit [Ping timeout: 246 seconds]
<{^_^}> firing: BuildsStuckOverTwoDays: https://status.nixos.org/prometheus/alerts
__monty__ has quit [Quit: leaving]
ixxie has quit [Ping timeout: 240 seconds]
teto has joined #nixos-dev
obadz has quit [Quit: WeeChat 2.7.1]
<domenkozar[m]> srk: around?
<bgamari> gchristensen, ping
obadz has joined #nixos-dev
bhipple has quit [Ping timeout: 250 seconds]
<{^_^}> firing: RootPartitionLowDiskSpace: https://status.nixos.org/prometheus/alerts
bhipple has joined #nixos-dev
<{^_^}> firing: RootPartitionLowDiskSpace: https://status.nixos.org/prometheus/alerts
ris has quit []