{`-`} has joined #nixos-ruby
<manveru> samueldr: thx
<qyliss> thank you samueldr!
<qyliss> I'm happy with how active this channel has immediately become, even if it's mostly just the two of us.
<manveru> just doesn't feel right to me to keep spamming offtopic with ruby stuff
<manveru> not many are interested in it anyway :)
<samueldr> -dev would have been fine I guess :)
<qyliss> I think having a dedicated place has encouraged us both to look at things more! Like this Bundler thing
<manveru> even i'm only interested for nostalgic and professional reasons :P
<manveru> samueldr: i guess so, it's been pretty quiet recently
<samueldr> (though not that there's anything inherently wrong in setting up a more specific location, except discoverability)
<manveru> that's fine with me
<manveru> samueldr: for a bit of context, we're working on a ruby.withPackages set
samueldr has left #nixos-ruby [#nixos-ruby]
samueldr has joined #nixos-ruby
<manveru> oh well :)
<samueldr> sorry, misclicked
<manveru> lol
<manveru> already thought you fled in panic
<samueldr> nah, actually instersted in seeing such a thing
<samueldr> I think I even asked in my first few weeks with nix about something like that, so we could get e.g. a cached build of nokogiri
<samueldr> just that one would be a seller :)
<manveru> jup
<manveru> nokogiri works already
<manveru> "hey"
<manveru> $ nix-shell --pure -E '((import ./. {}).ruby.withPackages (p: with p; [ nokogiri ]))' --run 'ruby -rnokogiri -e "p Nokogiri::HTML(%(hey)).text"'
<manveru> just the ruby-packages.nix has only a few packages so far, and the plan is to cover at least a substantial amount of the most-used ones
<qyliss> we _do_ technically have cached builds of some gems -- but no easily accessible derivations for them
<manveru> jup
<manveru> hmm
<manveru> if we had IFD...
<qyliss> IFD?
<manveru> import-from-derivation
<qyliss> ah
<manveru> we could keep the gemspecs around and resolve the needed dependencies during nix-build :)
<manveru> just have to keep every version of every gem in nixpkgs then
<manveru> that's probably a good project for NUR
<manveru> hmmm, even if i pin each line in the Gemfile to the latest version, bundler downloads all possible versions for a lot of gems
<manveru> looks like it goes depth-first, if it instead went breadth-first, it could parallelize the search at least
<qyliss> It is extremely annoying to me that I am pegging 1/48th of this machine's capacity
<manveru> :D
<manveru> i'm more surprised that it's still going
<manveru> but yeah, more cores doesn't really mean faster cores
<qyliss> Still occasionally downloading more versions of middleman too
<manveru> well, this is definitely not a viable approach
<qyliss> Yeah
<manveru> it's just that everything i can think of so far is a major pain to maintain
<qyliss> I think we need a faster bundler
<qyliss> We're asking a lot of it, so it doesn't have to be instant
<qyliss> but I'm still sitting here hours later
<manveru> even if bundler was instant, over time the package set would change drastically
<qyliss> I'm not sure that would be a bad thing?
<qyliss> As long as we watched for package versions decreasing, and curated the set to avoid that.
<manveru> well, for example this
<manveru> that's the latest github-pages, which would cause rails to downgrade to 4
<manveru> not to mention that people probably need more than one version of a gem
<qyliss> github-pages shouldn't be in the set -- it's not a library
<manveru> hmm
<qyliss> and if people need specific versions, they should use Bundler and Bundix
<qyliss> As I see it, this is really for scripts and stuff where you just don't care about the version
<qyliss> I see this as Nix's bundler/inline
<manveru> aye
<manveru> well, i'll just keep removing gems until i have a set of latest packages that's compatible with latest rails
<manveru> i think we can automate that too
<manveru> just need to have one fixed point around which we keep removing stuff that doesn't fit
<qyliss> I think it should be manually curated
<manveru> that's the pragmatic solution :)
<qyliss> Like, even if github-pages did work with latest activesupport, I still don't think we should have it
<qyliss> Because it's an application, and should have its own proper package.
<manveru> yeah
<manveru> i just didn't know that, haven't used it
<qyliss> just meant as an example
<manveru> and can't remove all gems that happen to have an executable
<qyliss> indeed
<manveru> so i guess manual it is for now
<manveru> and if someone wants a gem, they can make a PR
<qyliss> gems that have at least one dependent gem might be an interesting heuristic? Does anything depend on github-pages?
<qyliss> Yeah
<manveru> yeah
<qyliss> damn
<qyliss> ah well
<qyliss> ... why??
<manveru> anw, gotta make some dinner or my son starves to death
<manveru> maybe i'll play around with some better dependency resolution algorithms for our use-case with a bit of human intervention :)
<qyliss> :)
<manveru> but for now, can you assemble a list of gems you'd like personally?
<manveru> i only need 10 or so myself
<qyliss> For now, how about?: pry-byebug pry-doc activesupport rspec rugged
<qyliss> I don't actually regularly use all that many gems
<qyliss> I'm starting to experiment with generated bundler files
<qyliss> I'm seeing if I could make a JSON representation of a Gemfile.lock, which would be readable by Ruby and Nix
<qyliss> And could be turned back into a Gemfile.lock (and have a Gemfile generated from it) for Bundler.
<qyliss> Looking promising, though requires a bit of Bundler internals
<manveru> that'd be helpful yeah :)
<qyliss> This is going to be my Bundix 3 :P
<qyliss> It will have `bundix env` and generate a development environment
<manveru> because of the changed format?
<qyliss> And `bundix package` which will generate a package
<qyliss> And that will be it
<manveru> heh
<qyliss> And a changed CLI
<qyliss> I think the current one has grown too many options and is confusing
<qyliss> I still don't know what "magic" is :P
<manveru> i wish i'd never put that in...
<manveru> i don't even use any options anyway
<manveru> so yeah, i'm up for rewrites :)
<manveru> anything is more useful than what we got now
<qyliss> I like -l
<qyliss> Maybe -l can stay
<qyliss> Saves me having to add bundler to my nix-shell
<qyliss> but it's extremely simple
<manveru> that should really be what `bundix` does without any options
<qyliss> yes good point
<qyliss> it should have a --no-lock if you _really_ don't want to
<manveru> something like that :)
<qyliss> but I can't imagine that would be useful very often
<manveru> esp if you can generate the lockfile
<manveru> and gemfile
<qyliss> that's the plan
<qyliss> only thing you'll need to add to nixpkgs will be Gemfile.lock.json
<qyliss> An updated bundlerEnv will read that to figure out dependencies, and then run bundix to generate the gemfile / lockfile
<manveru> alright, had to push a new version of bundix
<qyliss> what for?
<manveru> it didn't support bundler > 2.0
<qyliss> oh yeah that was on my list :)
<qyliss> bundler 2 still isn't in nixpkgs iirc
<manveru> it is in mine :P
<qyliss> mine too :P
<manveru> was testing if it was faster
<qyliss> guessing it wasn't?
<manveru> nope
<manveru> anw, i got the list of gems pretty much done now
<qyliss> They didn't actually change anything for bundler 2 -- they've had the codebase support both for ages. it's just now they can start dropping support for stuff like old rubies
<manveru> aye
<qyliss> I'm waiting for https://github.com/NixOS/nixpkgs/pull/6925 before I add bundler 2 to nixpkgs
<qyliss> Because it's pretty painful to upgrade without it
<qyliss> since it probably forces you to build nokogiri or whatever outside of nix
<qyliss> so, don't want to be encouraging people to upgrade until we can give them a way of doing it that will actually work on NixOS
<manveru> ah :)
<qyliss> that change also needs an accompanying RubyGems patch
<qyliss> So it's a little bit away
<qyliss> it's taken a while -- glad I got started really quickly after Bundler 2 dropped
<manveru> not sure how that pr is related...
<manveru> are you sure that's the right link?
<qyliss> To update, you have to run `bundle update --bundler`
<qyliss> wait shit
<manveru> lol
<qyliss> I got the PR number right, but wrong repo
<manveru> damn that is a confusing conversation
<qyliss> i know right
<manveru> so... where to put all those new files
<qyliss> new files?
<manveru> well, i'm pretty sure i'll add pkgs/top-level/ruby-packages.nix
<qyliss> You shouldn't need too many new files, right?
<manveru> but where do i put the initial Gemfile, and the script to update it
<qyliss> Ahh
<qyliss> WWHD
<qyliss> (What would Haskell do? :P)
<manveru> plus i have to refactor withPackages to reuse more code :)
<manveru> and somehow make gemConfig work by passing it as override to ruby
<manveru> my final set of gems is only 129 now :)
<qyliss> not bad
<manveru> almost all from https://rubygems.org/stats are in there
<manveru> plus yours and mine
<qyliss> nice
<manveru> should cover a decent amount of use-cases
<qyliss> my `bundle lock`s are still running
<qyliss> i'm just gonna leave them because I want to know ho wlong they'll take
<qyliss> and if they'll finish
<manveru> you just wanna see on what it fails? :)
<qyliss> that too
<qyliss> I suspect the answer will be everything
<manveru> did you ever see the testing framework judson wrote in nix?
<manveru> i wonder if that's ever running
<qyliss> No?
<qyliss> Update: losslessy roundtripping every Gemfile.lock in nixpkgs to JSON and back!
<samueldr> \o/
<qyliss> And less than 200 LoC, which isn't bad considering it feels like I've reimplemented half of Bundler
<samueldr> IIUC the reasoning is that you need the json format (or implement a parser in nix) so there's no IFD, to resolve the right gems, right?
<qyliss> tomorrow I'll look at generating Gemfiles from Gemfile.lock.json too, but I think that's probably enough for tonight.
<qyliss> Yeah, exactly
<qyliss> Same reason we have gemset.nix now
<samueldr> yeah, that's good, checking I was groking the right parts :)
<qyliss> But using JSON instead of Nix means we can parse it from Ruby _in addition to Nix_. Which means we don't need the Gemfile and Gemfile.lock at all any more -- those can be generated.
<samueldr> nice, and eelco will not receive security notices from github since it won't be in the standard format :)
<samueldr> (I know, that's not the goal!)
<manveru> :D
<qyliss> lol
<manveru> i'd still like to run bundler-audit, but we can do that on the fly
<qyliss> Yeah. Would just have to modify the script to run bundix first.
<samueldr> considering it goes both ways, I guess it'll add a step to re-generate the gemfiles/lockfiles, hopefully easy enough
<qyliss> yeah
<manveru> i finished the refactor and cleanup: https://github.com/NixOS/nixpkgs/pull/61114/files
<manveru> now i just need to figure out how to make hydra build it
<qyliss> I'll have a look at that tomorrow
<qyliss> (the PR)
<qyliss> But, for now -- are the gems exposed as attributes, eg ruby.gems.nokogiri ?
<manveru> something like `ruby.withPackages (pkgs: with builtins; map (a: pkgs."${a}") (attrNames pkgs) )` at toplevel would probably work
<manveru> no
<qyliss> I would expose them like that
<qyliss> And then have withGems be nice syntax on top of that
<qyliss> for maximum easy of doing gem things, whatever they may be
<manveru> i'll try :)
<qyliss> and hydra will build them
<qyliss> thanks :)
<manveru> yeah
<manveru> also not sure all those gems even build atm
<manveru> seems like at least mysql gem is broken
<qyliss> btw, if anybody feels like looking at my Ruby devdoc PR that'd be rad
<manveru> i did... and tried it... seemed fine
<manveru> i don't have merge bits :)
<qyliss> you feel okay about the rbconfig hack?
<qyliss> manveru: you're still encouraged to review ;)