<samueldr>
ouch, I think I found an issue with github actions and their implementation: /nix/store is not kept between runs
<samueldr>
not between runs, between actions in a same workflow run *
<samueldr>
and imagine one wants to upload result/artifact from a non-nix-aware action, it can't be done through sneakily mounting overlayfs or copy/closure shenanigans
<samueldr>
I'm thinking that the more practical approach is to "unwrap" all links to /nix/store if the goal is to use the outputs for further processing outside the nix ecosystem, but I'm fearing it could leave symlinks around
<samueldr>
(and feeling this would be a huge hack :/)
* colemickens
looks at NixOS support on builds.sr.ht
* samueldr
looks at the userbase of github
<samueldr>
like, I'm not a fan of proprietary and vendor lock-in
<samueldr>
but if it would have worked neatly it would have been neat :)
<nh2>
Second, I don't understand what you're saying with "when ghc roams you to another thread": The things you quoted seem to all talk about _the tracee_, but the success of my program is dependend on whether or not I compile the _tracer_ with `-threaded` or not.
<nh2>
clever: 2 things:
<nh2>
First, you're right, the strace man page says `Note that -p PID -f will attach all threads of process PID if it is multi-threaded, not only thread with thread_id = PID.`, so what I said isn't true (the man page says that `-p` attachess all threads of the process even without `-f`, but I'm not using `-p).
<clever>
nh2: i think the attachment is for a given tracer and tracee thread pair
<clever>
nh2: so when the -threaded runtime moves your haskell thread to another host thread, it breaks
<nh2>
clever: ah, I read that differently: "in a multithreaded process, every thread can be individually attached to a tracer", it sounds to me like they are only talking about the multi-threadedness of the tracee
<clever>
i think that implies, that every thread, could even be traced by an entirely different tracee
<clever>
tracer*?
<clever>
which reminds me, the distinction between thread and process isnt as hard as you may think
<clever>
[clever@amd-nixos:~/apps/nixpkgs/pkgs/build-support/fetchurl]$ ls -l /proc/self/task/
<clever>
behind the scenes, every "thread" has a pid#
<clever>
and the only difference between threads and pids, is what they share, which is defined by flags to clone()
<nh2>
yes, that implication I understand from that sentence too, but that doesn't seem to imply what you said before directly
<colemickens>
Imagine if they just used nix
<clever>
if you abandon pthread, you are free to make "threads" that share nothing and behave more like procs
<clever>
and you are free to make procs, that share everything
<nh2>
clever: also, I've tried wrapping the whole business into `forkOn 1`, and it doesn't seem to fix it. But that may be because I use `foreign import safe` for `ptrace()`, so it can move each foreign call to a separate thread
<nh2>
clever: yes, it's all pretty much just clone() under the hood, both processes and pthreads
<clever>
nh2: ive also run into issues with the scheduling signals
<clever>
the ghc runtime will fire a lot of signals off, at regular intervals, to context-switch the threaded runtime
<clever>
that causes most syscalls to fail with EINTR or related
<clever>
however, some userland libraries treat that as a fatal error
<nh2>
clever: yes I considered that, I've turned all of that stuff off with `RTS -I0 -V0 -C0`, doesn't seem to help
<clever>
this allows you to block those signals, just for the ffi calls
<nh2>
clever: despite me not getting the same implication as you from the man page, your idea of the thing not being traced by that process is still quite likely.
<nh2>
I've just put in a couple more debug prints and can see that after the ptrace() has failed and says "No such process", a `ps -p THEPID` still shows the tracee as existent
lassulus has joined #nixos-chat
<nh2>
so if it still exists, it probably means it's not attached somehow
<clever>
i think you need to do all tasks against a given thread, within a single bound thread, via runInBoundThread
<nh2>
clever: `runInBoundThread` sounds good, I will try that. I will also try `forkOn` in combination with making the foreign call unsafe, so that no new thread is created for that
<clever>
and looking back at that old code, i think i'm using runInBoundThread wrong
<clever>
and its creating a new bound thread for each ffi call
<clever>
but pam wasnt using TLS, so it didnt matter
<nh2>
clever: `runInBoundThread` seems to fix it for me though, great find!
drakonis has joined #nixos-chat
<nh2>
clever: yep confirmed, `unsafe` + `forkOn 1` also fixes the problem. In fact, `unsafe` was what the authors of `linux-ptrace` used, but I changed it to `safe` so that it can't block the runtime. But `runInBoundThread` + `safe` is the right solution, in my opinion
<clever>
yeah, that sounds best
<nh2>
clever++
<{^_^}>
clever's karma got increased to 91
<nh2>
clever: now we just have to find why it is this way. So your interpretation of the man page / your follow-up comment is "Two tracees can have different tracer processes. Because in many situations, 'processes' can be read as 'threads', this can also be understood as 'two tracees can have different tracer _threads_'. That implies that running ptrace() from a thread that's not the tracer thread gives ESRCH", right?
<nh2>
I am very very happy we got this working, I've been investigating these nondeterministic failures for a while
<nh2>
clever: also fascinating was: When tracing long-running programs, it either failed within the first couple seconds or never, suggesting that it really depended on GHC's `safe` FFI thread pool spawn timing
<nh2>
`runInBoundThread` also immediately fixed the problem of Ctrl+Z and `fg` resulting in an immediate error (this probably triggered an OS thread switch)
<clever>
nh2: let me take a look at the kernel source...
<gchristensen>
what the hell this python program Iwant to install comes with its own patched version of python
<gchristensen>
"The self-extracting .exe file gives a virus message for some virus scanners. If you do not want this, use the .zip file!" hilarious, definitely no
<MichaelRaskin>
If SFX EXE gives a message on some virus scanners, and ZIP doesn't, I would actually blame virus scanners
<gchristensen>
quite possibly so
<gchristensen>
but I'm sure as hell not installing your sketchy patched (but undocumented on how) python
<MichaelRaskin>
I wonder if it is even actually patched
<MichaelRaskin>
Why is there pain seeping from the edges of this diagram?
<gchristensen>
that is the visual component of RSI
lejonet has joined #nixos-chat
<infinisil>
Ohh I just had a great idea: To make a browser extension that allows you to easily install a nix package on a button press when you're on a projects homepage, as given by meta.homepage!
<infinisil>
How cool would that be!
<MichaelRaskin>
The dual of the old single-click installers, in a way
<infinisil>
Would have to show a list of packages to install
<infinisil>
If there's multiple
<MichaelRaskin>
And if there are none, check for same-domain homepages, or maybe even parent domain if there are few
<infinisil>
Or just use "${pkg.meta.homepage}.*" as a matching regex (with appropriate escaping)
<infinisil>
(Would be hard to make an index out of that then though)
<gchristensen>
back in ye olde nix 0.8 days, there was a nix-install-package and application/nix-package mime type association
<gchristensen>
some software licenses say you can only run the software on one single computer. if that one computer is VM and it sometimes strts of hostA and sometimes starts on hostB, is that following the license?
<MichaelRaskin>
probably note, although the wording matters
<MichaelRaskin>
I wonder if a VM in a SSI would count.
<gchristensen>
SSD?
<gchristensen>
I*?
<MichaelRaskin>
Single System Image
<MichaelRaskin>
Unfortunately, nothing like that ever reached mainline Linux, and out-of-tree projects tend to die eventually
<gchristensen>
I wonder how that plays out w.r.t. live-migratable VMs and enterprise databass
<MichaelRaskin>
The idea is that you have a single Linux kernel that runs on a non-uniform memory access machine with some busses that happen to be Ethernet links
<gchristensen>
huh
<MichaelRaskin>
If you cut stuff into VMs and can live-migrate then indeed you don't need SSI
<MichaelRaskin>
If you have a multi-threaded application well-optimised for multi-socket NUMA, and a small rack of servers, you might prefer SSI
<MichaelRaskin>
But SSI is complicated, so yeah
sondr3 has joined #nixos-chat
sondr3 has left #nixos-chat ["WeeChat 2.2"]
lassulus has quit [Ping timeout: 244 seconds]
drakonis has joined #nixos-chat
drakonis_ has quit [Ping timeout: 255 seconds]
drakonis has quit [Read error: Connection reset by peer]
__monty__ has joined #nixos-chat
lassulus has joined #nixos-chat
<lejonet>
MichaelRaskin: what, distributed systems are hard? :O :P
<MichaelRaskin>
lejonet: actually, no, the problem is supporting and keeping track of all the stupid interfaces.
<MichaelRaskin>
The distributed part is already implemented for NUMA
<andi->
I heared Oracle treats all the CPUs you could eventually migrate to as part of that "one system". YOu would have to licenses ALL the CPU cores etc.. It really depends
<lejonet>
MichaelRaskin: true
<lejonet>
The whole license model for stuff that does per-core/per-socket is really... annoying
<MichaelRaskin>
I think AMD hinted that EPYCs are very cost-effective for per-socket licenses
<lejonet>
Yeah, mainly due to the large amount of cores/threads per socket possible with epyc (and the fact that they have 64 PCI lanes per CPU, so you don't need a tonne of CPUs/sockets for the PCI lanes if "all" you need is PCI lanes for like accelerator cards/GPUs)
<samueldr>
licensing per cpu pin when?
<MichaelRaskin>
Given around half of them are ground, it could lead to interesting designs to make then count as a single pin
lassulus_ has joined #nixos-chat
<lejonet>
license per hz...
<lejonet>
:P
<MichaelRaskin>
Per Hz of what?
<lejonet>
of the CPU
<MichaelRaskin>
Expect OoO machines with low clock frequency and low core count but insane operations per cycle in the best case
<samueldr>
maybe we need a benevolent licensor who figures out the best licensing scheme to push development of CPUs a specific way to avoid licensing fees?
<lejonet>
MichaelRaskin: Mhm
<lejonet>
I honestly don't understand the rational of licensing per socket/core or similar, like sure, I can understand licensing per feature or machine, but per socket? C'mon, that is clearly just for more moniez and nothing else
<MichaelRaskin>
This started with Oracle, whose business model around Java is best described as staying just short of fraud provable in court
<samueldr>
make your software $quality whenever you use more $hardware_thing and rake in more $profits?
<lejonet>
Indeed
<lejonet>
MichaelRaskin: I'm not surprised
<lejonet>
Because sure, for you to get better performance from software in a machine that has like 2 or 4 sockets for CPUs, the software has to be better, but its not like the improvements in the software is tied to the actual amount of sockets, but more of how good the code is and how well they separate workloads into "compute packets" so that you can fed whatever computational unit you use the most efficient
<MichaelRaskin>
Well, that's called price discrimination
<lejonet>
it just depends on so many more factors than "just" the hardware, that tying the license to what hardware or amount of hardware you is just greedy
<lejonet>
s/hardware you/hardware you have/
<MichaelRaskin>
Its product software business, of course it is greedy
<MichaelRaskin>
I wonder, do x86 CPU provide a nuclear option for proprietary software EOL? This new generation of out-of-order execution mangles our code more than was possible at the time of sales, so running the software there is creating unauthorised derived works!
<lejonet>
xD
<MichaelRaskin>
Same question about visible-source code and new compilers.
<lejonet>
Especially with high optimisation levels from the compiler
<__monty__>
Pretty sure they don't count as derived works because the compiler isn't applying creative thought.
<MichaelRaskin>
There is this legend that there are non-high optimisation levels left. It's a case of dangerous optimism — even at -O0 there is a lot that still happens to code
<MichaelRaskin>
__monty__: even if that speciism is still legally valid, be careful if you add more than three compiler flags
lassulus_ has quit [Quit: WeeChat 2.2]
endformationage has joined #nixos-chat
lassulus has quit [Remote host closed the connection]
lassulus has joined #nixos-chat
<MichaelRaskin>
Looked at NixOS Synapse expression. What, no option for LD_PRELOAD of jemalloc?
<MichaelRaskin>
I might instead admire these people for their inventiveness that leads to even noticing the Python bitness
<MichaelRaskin>
Ah, so the bitness really comes from NatLink
<gchristensen>
natlink is a bunch of python
<gchristensen>
plus other stuff
<gchristensen>
an OS where you can merge diverging branches would be fascinating
<gchristensen>
"in this one I ran updates" "in this one I installed XYZ" -> merge
<MichaelRaskin>
JUST NO
<tilpner>
So... NixOS with git-managed nixos-config?
<MichaelRaskin>
Well, if the question is about running updates as something that requires merging…
<drakonis>
gchristensen: yes please.
<gchristensen>
well in this case I'm wishing for it for windows10
<MichaelRaskin>
drakonis: did you even notice that the idea never includes «correctly» anywhere in the description?
<gchristensen>
I'm just very annoyed by Windows :P
<gchristensen>
and snapshots are a great start, but the linear progression is not sufficient
<MichaelRaskin>
Given how much !!FUN!! W10 can provide via mere updates, merging the states should be worth a full-feature movie.
<MichaelRaskin>
Snapshots are a save-load mechanism for when a party member dies, nothing more.
<MichaelRaskin>
HMMM. fMBT-based automation + snapshots might even work…
<gchristensen>
let's just say I've had a lot of party members die
<MichaelRaskin>
First victim: sanity
<MichaelRaskin>
Second victim: hope in humanity
<gchristensen>
love that windows update fails to apply updates literally to a brand new install
<MichaelRaskin>
Look, from time to time they manage to remove user files during an update. _Most_ of the time they actually just move them in an unguessable location. There are also _other_ times.
<averell>
at least the 18.09 update was completed 1-2 weeks ago :)
<averell>
maybe you'd have less trouble getting natlink to work with wine or something
<MichaelRaskin>
What takes less: Graham learning to set up this horror, or Mozilla DeepSpeech becoming usable?
<joepie91>
[21:52] <gchristensen> an OS where you can merge diverging branches would be fascinating
<joepie91>
reminds me of pijul
<joepie91>
(not an OS unfortunately)
pie___ has quit [Quit: Leaving]
<joepie91>
so... branchnix when? :D
pie___ has joined #nixos-chat
<MichaelRaskin>
Applying this approach to OS updates is likely to provide you examples of _all_ of the semantic traps with clean merges
<gchristensen>
once DeepSpeech gets close enough to usable, Nuance will destroy it
<averell>
it looks cool. too bad there's no nix package
<gchristensen>
for DS?
<averell>
mozilla can put up a few lawyers, no?
<averell>
yes
<MichaelRaskin>
Destroying Mozilla is generally not considered a wise plan
<infinisil>
joepie91: Wait what's the problem with NixOS + git?
<joepie91>
infinisil: Git is... Not Very Good at handling merging of divering branches without human intervention
<MichaelRaskin>
Lawyers might not be the only problem in the process…
<gchristensen>
well a few lawyers probably won't do it given nuance's long history of experience destroying companies and projects with its deep, rich patent portfolio
<joepie91>
infinisil: one of the premises of Pijul is that it's much better at that
<infinisil>
Ah
<MichaelRaskin>
Mozilla can just donate the project to FSFE
<infinisil>
joepie91: So NixOS + pijul should work better?
<joepie91>
infinisil: assuming the promises of pijul hold true, yes :P
<MichaelRaskin>
And they cannot
<MichaelRaskin>
I mean, it is easy not to do an enumerated list of stupid stuff Git does
<MichaelRaskin>
But there are cases where VCS has no reason not to accept the obvious clean merge, and the merge makes no sense
<joepie91>
this is also true for Git, though
<MichaelRaskin>
Git just doesn't have design for any of its layers, so it is basically unpredictable, who cares about git
<MichaelRaskin>
I even believe Pijul can be always not worse than the best of Monotone, Bazaar, Mercurial
<gchristensen>
high praise
<MichaelRaskin>
Well, they did invest into merge design!
<MichaelRaskin>
(I do not expect their actual history tracking to be always better than the worst of Montone, Bazaar and Mercurial!)
<__monty__>
Fedora silversomething and carbonOS kinda sound like "base an OS on git merging."
<gchristensen>
I meant my comment! :)
jackdk has joined #nixos-chat
<gchristensen>
I like pijul a lot, its only problem is the software
<MichaelRaskin>
The problem with merges is that there is an upper bound on how good you can be, and this upper bound is basically «not good enough»
<MichaelRaskin>
I think Pijul attacks the problem that is not even the main bottleneck
<manveru>
isn't pijul a fork of darcs?
<infinisil>
MichaelRaskin: Ideally we'd program in a language with provable correctness, then when merging you'd just have to compile it to see if everything still holds
<gchristensen>
manveru: not at all
<manveru>
used that back in the day... infinite merge times were so fun :P
<MichaelRaskin>
infinisil: too bad the merge also had to merge _the specs_ and messed _this_ up
<infinisil>
MichaelRaskin: Can this mess up though? If the specs change in some weird way, the implementation can't be correct
<infinisil>
And then this would get detected
<MichaelRaskin>
infinisil: ahahahaha
<infinisil>
?
<__monty__>
Pijul's basically a usable darcs 2.0 rn.
<MichaelRaskin>
You will have assumptions, in one form or another
<infinisil>
And if the spec changes exactly to match the implementation, everything is still fine because the spec literally defines correctness
<MichaelRaskin>
If you mess up your assumptions slightly, they can become unsatisfiable
<gchristensen>
__monty__: you've used it?
<__monty__>
I meant darcs 3/Camp.
<emily>
wasn't camp darcs 1.5
<infinisil>
MichaelRaskin: Example?
<__monty__>
Not used it no. That's what they seem to be promising.
<__monty__>
emily: Not according to the darcs.net/theory page I'm looking at rn.
<MichaelRaskin>
infinisil: well, _what_ your specification should claim?
<manveru>
guess i gotta give pijul a try then, loved the UX of darcs
<__monty__>
emily: Oh, Camp does sound like darcs 2.5 basically.
<gchristensen>
manveru: don't try it yet
<emily>
I remembered Camp coming before darcs 2 and darcs 2 incorporating some of its ideas, or something
<emily>
but this was long ago
<emily>
back when I was a haskeller and git didn't rule the world
<MichaelRaskin>
Darcs is annoying, you cannot succinctly refer to some observed states
<manveru>
gchristensen: why?
<gchristensen>
manveru: it is impossible to "just use" pijul. you will hit several bug-report-worthy problems in setup and regular use. the UX is not nice because of it.
<gchristensen>
unless you want that sort of proect!
<gchristensen>
but if you don't, it might leave a bad taste
<manveru>
heh
<infinisil>
gchristensen: I wish pijul just used github + git for itself
<gchristensen>
heh
<MichaelRaskin>
infinisil: wouldn't it be enough if they did stupid-dumos?
<manveru>
there's always fossil :D
<emily>
infinisil: do you also wish the subversion switching to git story was real? :p
<infinisil>
MichaelRaskin: No idea what that is
<MichaelRaskin>
stupid-dumps, sorry
<infinisil>
MichaelRaskin: No idea what that is either :P
<MichaelRaskin>
I mean, if you want to use git, you already cannot care about history
<MichaelRaskin>
So just a linear series with daily snapshots of the code would be enough
<manveru>
history is written by the winners... right?
<__monty__>
emily: What did you ditch haskell for? Rust, real life?
<infinisil>
MichaelRaskin: git still has a history? I'm not getting what you're trying to say
<infinisil>
And I do care about history
<infinisil>
And git is enough for that
<infinisil>
And it works well
<MichaelRaskin>
Erm
<MichaelRaskin>
Remind me when they stop assuming ext3 properties on ext4
<infinisil>
I've never had any problems because of that, seems like a niche case
<MichaelRaskin>
I had unfixable corrupted git repos
<MichaelRaskin>
Just after a few hard-reboots
<emily>
__monty__: homelessness :V more seriously, I like Rust but mostly wish the language I want to sketch up already had a compiler
<MichaelRaskin>
And speaking of caring about history — what is a natural way of syncing reflog in Git?
<infinisil>
Why would you want to do that
<MichaelRaskin>
Because I don't want to _think_ which part of the log was committed on staging and which on master?
<MichaelRaskin>
Or, I can put it another way: do you think reflog is useless?
<infinisil>
What log?
<MichaelRaskin>
Well, I have two revisions and I am trying to see what has changed. It would be very convenient to see which changes were done on which branch
<infinisil>
I still don't really see why you'd want to do that. You can check for each commit whether it is in a branch