<infinisil>
MichaelRaskin: Unfortunately we're stuck with git for the foreseeable future..
<infinisil>
And probably GitHub as well
<samueldr>
I'm not sure what was meant
<samueldr>
but I was thinking a bit, when the channel advances, NixOS/nixpkgs-channels/ updates
<samueldr>
would there be a value into also tagging the commit?
<samueldr>
I mean, I think there may be, I think there are tags which are "comitted", which would be the moment the channel advanced (or there about)
<samueldr>
maybe something like $jobset-$buildid or whatever the relevant hydra terminology is
<gchristensen>
git doesn't handle zillions of tags well
* samueldr
wonders what git handles well
lassulus_ has joined #nixos-chat
lassulus has quit [Ping timeout: 240 seconds]
lassulus_ is now known as lassulus
<gchristensen>
tracking files over time?
matthewbauer has quit [Read error: Connection reset by peer]
<samueldr>
there's hope then
matthewbauer has joined #nixos-chat
matthewbauer has quit [Ping timeout: 265 seconds]
jtojnar has quit [Read error: Connection reset by peer]
jtojnar has joined #nixos-chat
jtojnar has quit [Read error: Connection reset by peer]
jtojnar has joined #nixos-chat
jtojnar has quit [Remote host closed the connection]
<MichaelRaskin>
infinisil: re: GitHub — just wait till Microsoft buys it
<MichaelRaskin>
For different meanings of tracking…
<MichaelRaskin>
I think a branch with non-fast-forward merges could be an approximation
<MichaelRaskin>
(also, in -chat I care about DVCSes not _only_ for storing Nixpkgs, I just use Nixpkgs as an example… I want to increase the chance that someone will tell the news if something new and interesting appears, as I am still unsure if there is any feasible target for migrating my Monotone workflows)
brondif has joined #nixos-chat
jtojnar has joined #nixos-chat
brondif has left #nixos-chat [#nixos-chat]
Synthetica has joined #nixos-chat
jD91mZM2 has joined #nixos-chat
__monty__ has joined #nixos-chat
jtojnar has quit [Remote host closed the connection]
jtojnar has joined #nixos-chat
<gchristensen>
I've tried darcs and pijul and neither of them could stomach nixpkgs
<MichaelRaskin>
darcs cannot stomach even much smaller things
<MichaelRaskin>
By now I am not even sure I can say that git can stomach nixpkgs… git remote update on HDD is horribly slow, and I cannot talk git into repacking in a way that would allow throughtput-bound reading of the pack (I have enough RAM for disk cache)
<jD91mZM2>
I have git in my prompt and sometimes even the `git status` function ran by that is so slow that I don't get a shell prompt for several seconds
<jD91mZM2>
Perhaps the nixpkgs history should be automatically truncated after a few years
<gchristensen>
that hasn't to do with history
<gchristensen>
that has to do with checking the status of 18,300 files
<jD91mZM2>
Oh, right
<infinisil>
jD91mZM2: Can recommend getting an asynchronous git status prompt plugin instead
<joepie91>
so, uh, a bizarre realization: I've been looking at Lidl's economics behind their nonfood products (powertools, kitchen appliances, etc.) with somebody else, and we're strongly suspecting that Lidl is being used as a testing grounds for new/experimental features and products by larger brands like Bosch
<joepie91>
which seems to be the only possible explanation for why they appear to be selling complex machinery with reliable parts at essentially BOM cost, with uncanny similarities to particular brand-name products, and with weird handy new features that only appear in brand-name products a few months later...
<joepie91>
which would be a rather impressive way to do user testing :P
<jD91mZM2>
infinisil: How would that work? I still can't get the prompt printed before git status finishes
<jD91mZM2>
I've considered adding a timeout to it but libgit2 doesn't let you do that cleanly it would seem
<infinisil>
jD91mZM2: Not using it right now, it might only work with zsh
<infinisil>
I think it might be something with a mkfifo/writing the status to a file and have the prompt read from that
<joepie91>
I need to do a bit more research, see if I can verify this hypothesis
<joepie91>
but it would explain *so much*
<sphalerite>
how does doing that sort of analysis occur to you? x)
<joepie91>
sphalerite: I'm a hacker, things that make no sense on the surface stand out to me :P
<joepie91>
Lidl's economics look impossible, so I've had them in the back of my mind for a while
<sphalerite>
I suppose
<joepie91>
been talking to a few people, gathering some data
<joepie91>
noticing some patterns, like the novel features and products that appear at big brands months later
<joepie91>
in particular, they often seem ahead of the curve on new types/styles of products becoming popular, which is odd for a discount supermarket
<joepie91>
they just operate totally unlike any of the other low-budget non-food sellers
<joepie91>
so I can't curb my curiosity as to why that is :D
<joepie91>
sphalerite: it also explains why they have a rather insane 90-day no-questions-asked return policy
<joepie91>
which, *especially at this budget*, makes it seem even more economically impossible
<sphalerite>
riiiight
<sphalerite>
that's a cool model
<joepie91>
yeah, but it makes no economic sense :D
<sphalerite>
well if your suspicion is correct it does
<joepie91>
90 day no-questions-asked free returns with full refund, no return/restocking fee, and they'll even come and pick it up for you if you ask
<joepie91>
well yeah
<joepie91>
the only way this makes sense, is if they're more interested in feedback than in money
<joepie91>
then you'd want to incentivize people to keep using it for a while, and only return it later
<joepie91>
to get long-term feedback
<joepie91>
it would be rather impressive though, basically setting up an entire supply and service chain just to do user testing 'in the wild'
<sphalerite>
well it benefits all parties involved mostly, no?
<MichaelRaskin>
Well, if they already carry Bosch, the marginal setup might be reasonable
<sphalerite>
win for manufacturers, win for lidl, win for consumers (most of the time)
<MichaelRaskin>
Cheaper than the brand risk, I guess.
<sphalerite>
I don't think lidl has brand stuff?
<MichaelRaskin>
Bosch has
<MichaelRaskin>
Lidl… well, I guess they need to keep the «cheap-but-OKish impression», but if the safety is done properly, 90-day-returns cover the rest
<joepie91>
sphalerite: they do carry some brand stuff, but not Bosch (at least not branded as Bosch)
<joepie91>
sphalerite: and it would benefit all parties, sure, but even then it must have taken a lot of investment to get this setup going
<joepie91>
it's far more expensive than user testing in controlled groups
<MichaelRaskin>
Just because of scale, though
<MichaelRaskin>
It might be cheaper per-unit to organise than controlled testing
<joepie91>
that is possible
<joepie91>
that would also explain why they'd want to outsource the logistics to a party that's already operating at scale, namely Lidl
<MichaelRaskin>
Exactly
<joepie91>
if they had to set up their own distribution channels it'd probably be more difficult
<MichaelRaskin>
Logistics of supermarkets _is_ complicated and expensive, but it must scale — marginal cost of adding a product line must be low, or you have problems
<joepie91>
MichaelRaskin: this isn't just part of the regular supermarket logistics though
<joepie91>
the entire only sales channel is currently non-food only
<joepie91>
online sales channel*
<MichaelRaskin>
They do carry clothes there, too
<joepie91>
MichaelRaskin: so to summarize the sales model that Lidl uses... they have their physical supermarket chain, and then they have their online shop, and they basically operate separately as two separate parts of the company; however, once a non-food product is introduced (everything only has a single batch, no continuous sales of any product), it gets shipped out in certain amounts to every supermarket location where it's sold until it runs out,
<joepie91>
and a part of it gets shipped to the online shop warehouse where it gets sold until it runs out too
<joepie91>
there's no stock transfers between physical and online at all, logistics are two entirely separate organizations beyond the initial "get a batch of products and distribute it"
<joepie91>
you can't return online products in physical stores, nor vice versa
<joepie91>
(yet)
<MichaelRaskin>
How interesting
Synthetica has quit [Quit: Connection closed for inactivity]
<joepie91>
their FAQ does suggest that they are planning to change that, but right now it's totally separate :P
<MichaelRaskin>
But online part is still not only for power tools
<joepie91>
the warranty for non-food products is generally not handled by Lidl at all, but rather by the 'importer'
<joepie91>
which I suspect are just shell companies for user testing; Kompernass GmbH and such
<MichaelRaskin>
How they ended setting up a totally separate online presence is a good question, of course.
<joepie91>
MichaelRaskin: no, but the online part is limited to non-food [and wine]
<MichaelRaskin>
A.k.a. non-perishables
<joepie91>
there's a pretty clear distinction between food and non-food at Lidl, always has been
<joepie91>
food was continuous offerings, non-food was single-batch flash sales
<MichaelRaskin>
Yes, because if you mix up the logistics of persihables and non-perishables you are toast
<joepie91>
the latter mostly served to attract people to come buy the former, in the past
<joepie91>
MichaelRaskin: effectively-non-perishables like salt and sugar and such still count as food :P
<joepie91>
honestly I'm waiting for them to start delivering groceries, I've heard some noises that this is on the horizon
<joepie91>
but ~who knows~
<MichaelRaskin>
Well, storage conditions for flour and salt are still much stricter than storage conditions for even power tools
<MichaelRaskin>
(also, if you mess up flour storage, you are in for safety inspections, if you mess up power tool storage, you get some more returns than expected)
<joepie91>
well sure, but I don't think that's necessarily it; eg. bandages and other medical products often have fairly strict storage conditions too, and those are non-food
<joepie91>
I suspect the food/non-food split is mostly for historical reasons and it happens to match up pretty well with logistical requirements so why change it
<joepie91>
legacy, basically :P
<MichaelRaskin>
Yeah, I guess by the time medical stuff got there, the split was too stable
<joepie91>
MichaelRaskin: one other interesting data point is that the online returns aren't even really inspected or repackaged, they just end up on pallets of *stuff* and get sold off 'blind' by the pallet
<joepie91>
as misc returns
<joepie91>
and I've seen some Lidl-sold products pop up at a different supermarket a few months later, so...
<joepie91>
these supply chains are weird :P
<MichaelRaskin>
They are also even weirder than you think
<joepie91>
I learned that with the horse meat scandal, haha
<joepie91>
how it managed to go through like 11 intermediary parties before ending up on the supermarket shelves, most of which didn't do anything except for reselling it, I don't know
<MichaelRaskin>
Because spoiled and returned are different metrics, and margin is a one more metrics, and nobody cares about overall outcomes when they have KPIs…
<MichaelRaskin>
Choice between flash sale and write off of something that approaches expiry date might be… interesting
<joepie91>
MichaelRaskin: the 11 intermediary parties weren't the return/expiry supply chain though
<joepie91>
but the original one
<joepie91>
like, how it got from the butcher to the supermarket shelf
<MichaelRaskin>
That's true, but that's not even that surprising
<MichaelRaskin>
I mean, aggregation/statistical smoothing/re-chunking are quite natural things to value for a manager with a few KPIs.
<jD91mZM2>
infinisil: So basically I'd delay the git part of my prompt and when it's done reload it?
<jcrben>
andi-: did you ever end up getting typometer to work nix style?
<andi->
jcrben: nah, didnt try furhter
<andi->
so, what will we do after Microsoft has acquired GitHub?
<simpson>
Nothing?
<andi->
thats a valid response :)
<MichaelRaskin>
Setup a backup of issue data, maybe
<MichaelRaskin>
And wait until it is needed.
<andi->
well I had the discussion with a different group of people today.. that went in the opposite direction: move away as far as you can.
<samueldr>
the thing is, until something happens, it's extremely hard to know what will happen
<MichaelRaskin>
What's the risk, if we do backup the data?
<samueldr>
and in reality, all "microsoft situations" could happen microsoft or not
<simpson>
samueldr: Exactly. To me, MS isn't a worse owner than GH.
<samueldr>
I mean, nothing stops the current github to do all the nefarious scenarios
<andi->
I might be oldschool but for me that alread feels like an invasion. Never really liked that GitHub is basically a central commercial piece for opensource software..
<samueldr>
it's not as if github was extremely open
<samueldr>
and yes, my feelings for github goes in that direction: it's already a walled garden
<samueldr>
microsoft could be an even better manager for a walled garden, who knows!
<andi->
What are the reasons they have to buy the software? Love for OSS? Trying to acquire a profitable company? Replacing/changing their software porfolio because their on-premise/paid software stack (team foundation foo?) doesn't sell anymore?
<andi->
I hope I am wrong..
<samueldr>
it could be as simple as: thing produces money, they want money
<samueldr>
could be as nefarious as wanting to turn it into an azure product
<andi->
I am not so sure if GitHub makes that much money. It is just well known/accepted & being used.
<__monty__>
What's this about microsoft acquiring github?
<samueldr>
yes, it wasn't a researched fact, but a supposition where there was no nefarious goals
<samueldr>
I only want to say: let's wait and see for all the doomsday scenarios, it won't be productive
<samueldr>
and as for backing up, it should happen anyway IMHO
<samueldr>
ms or not, no one can know what's going to happen
<andi->
yep
<MichaelRaskin>
That's just the question if we are just looking for an excuse for «issues default» or not
<andi->
how do you mean?
<simpson>
I don't think that "wait and see" is a good strategy. In general, hope is not a good strategy. We should either backup the issues or be okay with losing them. (I'm okay with losing them.)
<samueldr>
do you mean there is no backup strategy in place for issues/PR?
<samueldr>
scary!
<andi->
I think the damage done if we would loose access to all currently outstanding PRs would be limited... I mean they have a tendency to rot. We currently have >640 open PR. It's been a year since I've seen that being lower then 500?
<andi->
so a lot of work is there but probably bitrotting. The valuable pieces are probably already decentralized
<samueldr>
I'm not worried about the actual open PRs in waiting
<samueldr>
but more about the discussions in the issues and PRs
<samueldr>
sometimes used as references
<joepie91>
*if* the rumour is true, then the #1 reason Microsoft would want GitHub would probably be platform integration
<andi->
joepie91: thats what I was trying to hint up there.
<samueldr>
the issues and PRs history often explains why things are as they are
<joepie91>
which can take any form from "hey you can login with Azure now" to "your project now lives in the Microsoft ecosystem"
<andi->
samueldr: true
<MichaelRaskin>
For issues, I sometimes feel like some people would consider an externally-caused reset a plus
<andi->
joepie91: I think they may focus integration in their IDE.
<andi->
like they did with the TFS
<samueldr>
I would be fine with expiring issues that are over a treshold of inactivity
<joepie91>
samueldr: define 'expiring'
<samueldr>
closing
<joepie91>
okay, yeah, don't do that
<joepie91>
issues should be closed when they are either solved or ruled invalid
<samueldr>
let's rephrase
<samueldr>
manual review
<joepie91>
not because they are merely 'inactive'; because that tells you nothing about whether the issue still exists, whether it was one in the first place, etc.
<samueldr>
with a system
<joepie91>
sure, I'd be all for triaging :)
<samueldr>
so not "ah I see it's 6 months since last activity close thanks"
<joepie91>
right, okay
<samueldr>
but more "hey, is this still an issue, is there something actionable?"
<joepie91>
too many projects do exactly that...
<joepie91>
yeah, that's fair
<samueldr>
right now I have seen issues, even MINE that were open but fixed :)
<joepie91>
like, triage the issue: either 1) the issue is reproducible and needs to be resolved, so it stays open, 2) the issue is invalid, so it's closed, 3) it's unclear, so you ask for elaboration, and then a) if you get a response, goto 1 or 2, b) if you don't, close after a while because not actionable
<joepie91>
(I count 'already solved' under 'invalid')
<samueldr>
joepie91: basically what I was thinking
<joepie91>
samueldr: I suspect it could be beneficial to do sprints of this, collaboratively, too
<samueldr>
and closed shouldn't be a final state, it could be re-opened if e.g. the user missed the questions
<joepie91>
yeah, agreed
<joepie91>
I wonder if people would be interested in a meatspace meetup for triaging bugs collaboratively for a few hours :P
<andi->
I was thinking about something along those lines earlier... Having a hackathon a few times a year.
<joepie91>
once I'm a little less overscheduled, I could probably look into organizing one of these at the hackerspace in The Hague, if there's enough interest
<andi->
I friend of mine is currently organizing an OpenBSD hackathon at the loxal university. They seemt to be friendly enough with offering rooms/infra.
<joepie91>
I suspect they'd be fine with hosting this
<joepie91>
(one of the people who would make that decision, is very much in favour of 'traditional' hackathons, ie. collaboratively fixing OSS stuff :P)
<andi->
btw I did grep a bit through the nixpkgs git log.. turns out there are "only" 4k mentions of github there. We apparently had a package called haskell-github-backup ;)
<MichaelRaskin>
Maybe just download all the issues (by number) as pages?
<andi->
yeah
<andi->
that would probably a sane fallback
<andi->
not sure how we handle comments on commits..
<MichaelRaskin>
Would it be even noticable relative to the binary cache size?
<andi->
probably not..
<joepie91>
scraping the API would probably be a better idea
<andi->
but the crawling would be slow/rate limited?
<andi->
joepie91: but then you also have to write a presentation for that...
<andi->
and it might render differently..
<andi->
etc..
<joepie91>
I don't think github's API is unreasonably rate-limited?
<MichaelRaskin>
I have a suspicion that API rate limiting is more rate limited than non-logged-in page scraping
<joepie91>
andi-: well sure, but are we archiving comments for historical reasons or for functional reasons?
<joepie91>
like, integrating past comments into a new system is really difficult if you have page copies
<joepie91>
much easier if you have the 'raw' data
<joepie91>
but for historical-archiving reasons you'd want page copies, a la wayback machine
<joepie91>
different usecases with different requirements :)
<andi->
ahh.. ofc in the best case we'd have the data als properly formatted blob of JSON or whatever..
<andi->
but just thinking simple here.. maybe we just need a text copy for the start
<joepie91>
MichaelRaskin: more, sure. but what matters is whether it's rate-limited too much :P
<joepie91>
produces WARC files, which 1) are wayback-machine-compatible, and 2) store raw HTTP requests and responses, so you don't have issues with eg. wget overwriting or duplicating files on disk
<MichaelRaskin>
Frankly, uniformly-formatted Github pages can be asily parsed into data
<andi->
the question is does github already cooperate with archive.org?
<joepie91>
MichaelRaskin: there are a lot of edge cases in HTML parsing
<joepie91>
andi-: not publicly afaik
<MichaelRaskin>
Parsing HTML generated by a single generator is usually easy
<joepie91>
MichaelRaskin: you'd be surprised :P
<joepie91>
in a dataset as big as that of github, you'll suddenly find out about all the edge cases for stuff like "oh this comment doesn't have a date because we didn't store those back then"
<joepie91>
like, for example..
<MichaelRaskin>
Look, I subscribe to a lot of stuff by parsing the homepages and extracting links to updates
<__monty__>
Would microsoft really ruin github? They've helped the haskell community without ruining it so far.
<joepie91>
MichaelRaskin: yes, but that is *new* data, not historical
<MichaelRaskin>
Usually I download the backlog
<andi->
mhm.. maybe the issue is trivial: Is anyone subscribed to EVERYTHING on nixpkgs since it was created? ;-)
<andi->
That person should have a lot of emails
<joepie91>
MichaelRaskin: here's an example of what I mean: https://npms.io/search?q=libusb -- note how the bottom-most result does not have a date in the 'updated by'
<joepie91>
that's because the node-usb package predates most of the metadata that the npm registry now stores for packages
<joepie91>
most tools will break when encountering this package
<MichaelRaskin>
Well, (ignore-errors (select …))
<MichaelRaskin>
And don't care
<MichaelRaskin>
Hm
<MichaelRaskin>
Github UI doesn't show old notifications
<andi->
if I click on "all notifications" I see quiet a lot.. but only for about 30 days it seems
<samueldr>
I did once write a github scraper using the API for counting PRs
<samueldr>
the API is reasonably limited
<andi->
we can always spread the load amonst many API tokens
<andi->
the real question for me is: if I start scraping now will I finish? :D
<samueldr>
for all PR objects (so not comments) for nixpkgs I was able to scrape without limiting in one go
<samueldr>
(the worst was RAM usage since I made no optimizations and kept all in-ram)
<andi->
Looking at the RFC repo I wonder how difficult it is to catch all those in-line comments and discussions including the matching src snippet..
<joepie91>
figured I'd throw archivebot (http://dashboard.at.ninjawedding.org/3) at the nixpkgs issues, to see how long it'd take to archive all the pages in there; I expect the job to start somewhere in the next 5 hours
<joepie91>
(it's currently in queue)
<joepie91>
that'll at least produce a wayback machine copy