<samueldr>
hey, I was racking my brain a bit, but before I need to really make it work, anyone knows tricks relevant to figuring out the list of all commits not coming from github pull requests?
<samueldr>
there are a couple issues
<samueldr>
1. not all merges will be made through the github interface
<samueldr>
2. not all merges through the interface are forced to be a merge with commit message
<samueldr>
3. figuring out using branches from the origin pull/* magic probably breaks once a branch is deleted (unverified)
<infinisil>
How about just using the github api?
<infinisil>
Just get all commits from merged PR's and remove those from the list of all commits
<samueldr>
ah, if there is such an API, then I'm all set :)
<samueldr>
(and yes that's the kind of comment I was looking for)
<infinisil>
drakonis: Ah, yeah you should update the public key on github probably
<infinisil>
drakonis: That's on the github website in the settings somewhere, ssh public key
<drakonis>
don't think i've ever set that up
<drakonis>
oops i did not
<infinisil>
samueldr: What's that exactly?
<drakonis>
oh no
<drakonis>
i'm an idiot
<drakonis>
what i'm messing with, is the wrong repo
<infinisil>
Heh, it happens
<drakonis>
this is the master repo i'm fiddling with
<samueldr>
infinisil: it adds a commit, without any parents, with a README, then I can set it as the default branch so I don't show a stale master branch
<samueldr>
infinisil: I try to do it for everything forked
<drakonis>
turns out that my own copy is moving at turtle steps
<drakonis>
22 KB/s
<drakonis>
this is too slow :(
<drakonis>
by the time it is done fetching, it'll be tomorrow
<drakonis>
gchristensen, i'm curious, when did you leave tumblr?
<gchristensen>
February
<drakonis>
huh, neat.
<gchristensen>
oh?
<drakonis>
i only found out because eelco tweeted about joining the same company as you
<gchristensen>
ahh yeah :)
<drakonis>
to the future eh?
<drakonis>
is it normal for a fetch to run so slowly?
<gchristensen>
yeah, I didn't like working for Oath whose reason to exist is to sell ads to children basically
<gchristensen>
and I wanted to work on Nix* full time so here we are
<drakonis>
you mean easily influenced people
<drakonis>
but yeah
sir_guy_carleton has joined #nixos-chat
<samueldr>
oh, the github API tells us that author_association=>"FIRST_TIME_CONTRIBUTOR"
<drakonis>
huh, neat
<samueldr>
ah, it's shown on hover on the user's informations "Opened this pull request (their first in NixOS/nixpkgs)
<samueldr>
oh wow weird
<samueldr>
it also already has a merge commit made, with hash
<samueldr>
yeah, so from the API this is the commit from current tip of master and one of my PRs
<drakonis>
"Chaff Bugs: Deterring Attackers by Making Software Buggier"
<samueldr>
linux-sunxi is a much-forked repo too
<samueldr>
ah, my bad
<samueldr>
it's already a fork of linux/torvalds and share the same forks count
<drakonis>
um, wow, 3 years produced nearly 500k commits
lassulus_ has joined #nixos-chat
<samueldr>
they work on an entirely different level
<samueldr>
it's an interesting organizational work if you read on it
<ivan>
hah, nixpkgs has more forks than stars
<drakonis>
i've read up on how it works and it is fairly impressive
<drakonis>
but what does impress me is that the rate has ramped up
<samueldr>
haha ivan that's right
lassulus has quit [Ping timeout: 248 seconds]
lassulus_ is now known as lassulus
<infinisil>
Most commits in nixpkgs are probably just version updates lol
<drakonis>
well yeah :v
<samueldr>
infinisil: noted, I'll try to stat that out
<drakonis>
finally, it finished
<drakonis>
what's the next step
<drakonis>
finally happening
<infinisil>
drakonis: git reset --hard upstream/master if you want to reset your own master to upstreams one
<drakonis>
finally... it is complete, time to update some packages
<drakonis>
i did a git push to refresh my fork to avoid doing this again
obadz has quit [Ping timeout: 260 seconds]
Guanin has quit [Read error: Connection reset by peer]
obadz has joined #nixos-chat
drakonis has quit [Remote host closed the connection]
sir_guy_carleton has quit [Quit: WeeChat 2.0]
Myrl-saki is now known as shot-a-con
shot-a-con is now known as Myrl-saki
<joepie91>
mumble mumble.
<joepie91>
it's one of those days... I'm spending basically my entire morning writing tests
<joepie91>
lol
<ldlework>
joepie91: become funemployed and do what you want
<joepie91>
ldlework: well, I'm not writing these tests for an employer or client... :P
<joepie91>
it's for one of my own projects
<joepie91>
but automated tests are needed here
<joepie91>
so...
<ldlework>
joepie91: what's the project?
<joepie91>
ldlework: the best description I can come up with for it is a 'data modelling library for JS'; it has bits and pieces of different things... static typing (albeit runtime-checked), data validation / constraints, sort-of relations
<joepie91>
essentially a way to centrally specify what your data *should* look like, to a degree of granularity that includes custom validation logic
<joepie91>
with a 'type-safe' serialization aspect to it as well
<joepie91>
goal is to replace the validation libraries, function guards, data mangling, etc. that tend to be strewn throughout a JS project
<joepie91>
with centralized data definitions
<joepie91>
(but extending quite a bit beyond what you'd get with stuff like typescript/flow/etc.)
<joepie91>
anyway, automated testing is needed here due to the many subtle issues that could result in high impact on application reliability :P I'm essentially treating it as security-critical code, so it gets a shitton of tests
<joepie91>
ldlework: seems similar-ish, though in my case I'm opting for composable traits over inheritance, and I'm not certain that the purpose is exactly the same :P
<ldlework>
joepie91: my idea was to take the patterns that ORM's in Python were using to define database models, with associated validation and reflection, and provide that for arbitrary data types in your application.
<joepie91>
ldlework: I *am* taking some cues from RDBMS design, so in that sense it's similar
<joepie91>
but my implementation is also part-inspired by how Rust handles data + logic
<joepie91>
like, within the language itself
<joepie91>
ldlework: also, I recently discovered that Perl has value constraints natively :P
<joepie91>
subset Pincode of Str where /^ \d\d\d\d $/;
<joepie91>
my $adhoc where *.elems == 5;
<ldlework>
i haven't taken Perl seriously in a long time
<joepie91>
(examples I was given by somebody)
<joepie91>
also, Perl 6 specifically
<joepie91>
I still have issues with a number of the design considerations behind Perl 6, but at least it's a lot more sanely designed than 5 :P
<ldlework>
i'm sure perl has improved over time
<joepie91>
and it's a very different language than 5, it seems
<joepie91>
'similar in name only' type of deal
<ldlework>
joepie91: I'm in love with MLs at the moment
<joepie91>
ldlework: haven't had especially much time to look into them in detail
<ldlework>
i find them super elegant and expressive
<joepie91>
ldlework: I did experiment with a language of my own that has some parallels to MLs, but my main interest right now isn't in text-based programming at all, but rather visual approaches
* ldlework
hides in a bush.
<joepie91>
and to be clear, not the "anybody can program without any effort!" school of them
<joepie91>
but rather the application of general-purpose visual programming languages to real-world software projects by 'serious' developers, ie. developers who try to understand how things work and how to use them correctly :P
<joepie91>
to improve reliability and 'readability'
<ldlework>
joepie91: i have tried PureData and it is absolute madness
<joepie91>
very different from what I'm looking at :)
<joepie91>
I'm more thinking flow-based
<ldlework>
send me a screenshot
<joepie91>
heh, nothing to send a screenshot of yet, most of it is still in my head
<joepie91>
ldlework: I have some sketches somewhere but not much more than that, in terms of showable things
<ldlework>
ah well
<joepie91>
ldlework: most of my projects are 80% done in my head, over a long period of time, concurrently with a lot of other projects... and eventually once I feel that I've figured it out, an implementation starts materializing
<joepie91>
where programming is concerned, also usually involves a lot of passive observing of other developers and what they're doing / what issues they run into, reading about related topics, etc.
<joepie91>
so for most of the things I work on, there's just nothing to show yet because there's no implementation yet :P
<joepie91>
but there will Eventually(tm) be
<ldlework>
well best of luck then :)
<joepie91>
thanks :P
<joepie91>
I'll probably talk about it here once I have something concrete, heh
<joepie91>
current idea is to test out the visual programming idea in the context of a game development tool, as (budding) game developers tend not to be very tribal about their tools so long as they work well, and it's a reasonably well-defined subset of software to support
<joepie91>
so, seems like a good testbed
<joepie91>
anyway, brb
<ldlework>
visual game programming languages have been attempted
<joepie91>
ldlework: my goal isn't a 'game programming language', though; my goal is a *generic* visual programming language, with game development as the initial testbed
Myrl-saki is now known as bin_sh_otacon
Lisanna has joined #nixos-chat
pie_ has quit [Ping timeout: 244 seconds]
pie_ has joined #nixos-chat
<infinisil>
Lol, my AWS Polly speech synthesizer free period is over, just got my first invoice..
<infinisil>
I gotta pay a whole $0.01!
pie__ has joined #nixos-chat
pie_ has quit [Read error: Connection reset by peer]
pie___ has joined #nixos-chat
pie__ has quit [Ping timeout: 268 seconds]
obadz has quit [Ping timeout: 260 seconds]
obadz has joined #nixos-chat
bin_sh_otacon is now known as womenarejustadul
womenarejustadul is now known as Myrl-saki
Myrl-saki is now known as menareadultshota
menareadultshota is now known as Myrl-saki
<samueldr>
rate limits are a pain :(
biopandemic has joined #nixos-chat
drakonis has joined #nixos-chat
__monty__ has joined #nixos-chat
Sonarpulse has joined #nixos-chat
tazjin_ has joined #nixos-chat
tazjin has quit [Ping timeout: 256 seconds]
drakonis has quit [Ping timeout: 256 seconds]
tazjin_ is now known as tazjin
drakonis has joined #nixos-chat
drakonis has quit [Remote host closed the connection]
drakonis_ has joined #nixos-chat
Guanin has joined #nixos-chat
<samueldr>
oof all the responses from the pull + pull/commit API from nixos is ~778M in a sqlite database
<samueldr>
(with the actual json response saved, with some fields copied to query)
<samueldr>
though now I'm free from rate limits for doing fancy things :D
jtojnar_ has joined #nixos-chat
jtojnar has quit [Ping timeout: 264 seconds]
jtojnar_ is now known as jtojnar
<makefu>
samueldr: wow you scraped the whole history? what tool did you use? maybe you could quickly archive this via archive.org uploader
<samueldr>
makefu: ugly octokit code with manual restarting after rate limits
<samueldr>
right now it's not of archival quality
<samueldr>
(it's possible I lost data)
<makefu>
ah okay
<samueldr>
but this gives me a starting point to work out a POC
<samueldr>
then I will look into a safer, more archivable one
<samueldr>
I want this to be available for others too
<samueldr>
the database can, AFAICT, be safely continued
<samueldr>
as in, if I save the right information as a checkpoint, it's 100% possible to fill in the updated data
<samueldr>
with some caveats (like e.g. a user changing name and not showing up in newer updates wouldn't see its record updated)
<samueldr>
(and we don't get the full actual true history, just a snapshot of the history as github tells us at this moment)
<makefu>
i think this is fine
<samueldr>
I think too
<samueldr>
and my use case isn't archival, it's for data sleuthing
<makefu>
i was suggesting archive.org not for archival, but just for a somewhat reliable storage and sharing solution
<samueldr>
ah
kini has quit [Quit: bye]
<samueldr>
yeah, I also think it would be worthwhile to share the already dumped data
<makefu>
yep, exactly
<samueldr>
rate limiting from github is unhelpful, though it's really generous
<samueldr>
I think I restarted only 7 times
jtojnar has quit [Remote host closed the connection]
sir_guy_carleton has joined #nixos-chat
<makefu>
that is not too much, really
jtojnar has joined #nixos-chat
<samueldr>
yes
<samueldr>
that's why there's still a FIXME : automate waiting for rate-limiting errors
<samueldr>
the API even gives you a timestamp for when you can begin next asking!
<makefu>
just do an enterprise-loop around it (or until ... ; do sleep 30m; done)
<samueldr>
yeah, I had to manually update the page number :)