gchristensen changed the topic of #nixos-chat to: NixOS but much less topical || https://logs.nix.samueldr.com/nixos-chat
drakonis has joined #nixos-chat
andi- has quit [Remote host closed the connection]
andi- has joined #nixos-chat
drakonis_ has joined #nixos-chat
drakonis has quit [Ping timeout: 245 seconds]
endformationage has quit [Quit: WeeChat 2.6]
johanot has joined #nixos-chat
Jackneill has joined #nixos-chat
cransom has quit [Quit: WeeChat 2.4]
buckley310 has quit [Read error: Connection reset by peer]
buckley310 has joined #nixos-chat
<eyJhb> johanot: Are you ready for another round of being unproductive? ;)
<eyJhb> I have a head start, by going out drinking yesterday and then comming in at 10..
__monty__ has joined #nixos-chat
pie_ has quit [Ping timeout: 240 seconds]
psyanticy has joined #nixos-chat
Arahael has joined #nixos-chat
pie_ has joined #nixos-chat
<Arahael> I've finally installed home-manager! What's some great "dotfiles" that use it?
<etu> arahael: man home-configuration.nix
<etu> arahael: Great way to find options for home-manager :)
<Arahael> etu: I want to get ideas of say, how best to ensure I have one git configuration for work, and one for home.
<etu> arahael: Oh :)
<Arahael> Beautiful - thanks :)
<etu> And then I have the file: ~/.config/git/work_config on my work laptop where I override things, such as email and gpg and stuff :)
<Arahael> Oh, interesting... You use a condition for your git dirs to enable the work configuration, selectively.
<etu> Yep :)
<Arahael> I want this to be global, effectively - I dont' do work at home, and vice versa.
<etu> Hmm, I don't either do private stuff (except from that shared nixconfig) on my work laptop, but all my work projects are in the same folder so I solved it that way :p
<etu> I'm not sure which other conditionals git supports
<Arahael> Hmm.
<Arahael> Well, I use ~/src to store my work and personal repositories, in both environments. But I only check out work projeccts on the work laptop and vice versa.
<Arahael> Maybe I should have environment-specific home.nix files and do the symlink trick?
<etu> Could work! :)
<johanot> eyJhb: I'm actually kind of having a day of, because I've beeing "working" so much lately ;)
<eyJhb> johanot: then you can easily join me in not being productive!
<eyJhb> Or.. I have to play with some SQL Queries.. Which is no fun.. Especially not when they include intervals and shit
<johanot> "kind of" == once you even start planning a day off, systems/servers go haywire.
<eyJhb> Which IPs should I target johanot ?
* johanot considers..
<Arahael> etu: Does this look right? The idea is that work-specific configuration is set in work.nix, and I merge that with common.nix: https://gist.github.com/arafangion/46d161b7c48ed272e72cde2e061f6fce
<eyJhb> I have a GW IP that might be fun?!:D
<eyJhb> I just got a introduction to Splunk yesterday, it is actually quite "fun" :p
<johanot> eyJhb: can you possibly take out our surveillance please :D that could create some peace.
<etu> arahael: Hmm, I guess something can work. I would do "inherit config pkgs;" in the set sent to the import
<johanot> eyJhb: well.. if we can't take a casual hit on our gw, then it's not my fault :)
<Arahael> etu: ie, `{inherit config pkgs;};`? Nice. Would the work.nix config take priority?
<eyJhb> johanot: CASUAL?! How dare you ;) :p How does 10-20 Gbit sound? :D Don't think it will take it down, but could be fun :D
<johanot> eyJhb: I can drop packets reeeeeally fast if I need to :p but hey.. I don't manage the perimeter firewall, so it's not my problem anyway :D
<eyJhb> Hmm.. I wish I could say I had multiple IPs etc. buuut I really don't.. :p If you were evil you would tarpit it
<eyJhb> etu: how do you vet members of your underground base?
<Arahael> Hmm, I've set programs.git.userEmail but it doesn't show up when I do 'git config user.email'
<etu> eyJhb: We don't
<eyJhb> etu: so people just pay X, and then get access?
<etu> eyJhb: yeah, kinda
<etu> arahael: Have you done programs.git.enable = true; ?
<eyJhb> etu: have you ever had problems with that? :/
<etu> arahael: And do you have a .gitconfig in your home directory?
<Arahael> etu: Yes. And yes.
<Arahael> Well, yes, and very very probably yes, just checking.
<etu> arahael: Is that file a symlink to the store?
<Arahael> Yes. :)
<Arahael> Ah, and no, it's not a symlink to the store.
<Arahael> However, when I did a home-manager switch, it had no warnings.
<etu> arahael: Because home-manager put the files in ~/.config/git/config, and I guess that git doesn't look there if you have ~/.gitconfig
<Arahael> etu: Ah!
<Arahael> Next problem... It seems that the common.nix *overrides* the environment specific stuff.
<Arahael> How do I make it do the other way?
<etu> put the common before // to merge it in the other direction
<etu> I think should work
<etu> > { a = "a"; } // { a = "b"; }
<{^_^}> { a = "b"; }
<Arahael> Hmm... Ok, I think I'm going to have to be funkier. I thought // does a *merge*.
<etu> > { a = "b"; } // { a = "a"; }
<{^_^}> { a = "a"; }
<Arahael> See, I have common do the programs.git.enable = true.
<__monty__> arahael: mkMerge?
<__monty__> I *think* that recursively mkMerges?
<Arahael> __monty__: I don't see that in the nixpkg's manual?
<Arahael> But a 'recursive merge' sounds like what I want.
<Arahael> Or am I just overcomplicating stuff?
<__monty__> It's in lib/modules.nix
<Arahael> Yes, I don't know how to use modules yet, I think I'll keep this in a form that doesn't need recursive merging.
pie_ has quit [Ping timeout: 265 seconds]
drakonis has joined #nixos-chat
drakonis_ has quit [Ping timeout: 245 seconds]
AstraAdria4Ari has joined #nixos-chat
<eyJhb> If I have 18.000.000 MySQL records, that I need to group by date, and I only have a timestamp to go from. What would be the best approach?
<eyJhb> I am thinking that I need to add a additional field, that has the date...
<gchristensen> a unix timestamp?
<eyJhb> Yup, e.g. `1468515851`
<Arahael> eyJhb: Does it matter if you sort by time within those dates as well? Because if it doesn't matter, just sort by timestamp.
<Arahael> Numerically, of course...
<gchristensen> cool, that is all you need
<etu> eyJhb: While timestamp >= 1231232 and timestamp <= 12312312
<eyJhb> Ehmm.. I should clarify
<eyJhb> I need to count all the records that happened X day, in the form of group by, as I would take a range of e.g. one month
<eyJhb> So each returned row would be a date (2018-12-31), which has a summaries count of how many rows were on that date/day
<eyJhb> Does that make better sense?
<Arahael> eyJhb: Yes, and that makes for a Hard Problem.
<eyJhb> Yeah...
<eyJhb> Or not hard, but time consuming :p
<Arahael> eyJhb: What timezone are these in, for starters?
<eyJhb> UTC+2, but that isn't that important, as they will never be different
<eyJhb> So that can be hardcoded, no problem
<Arahael> eyJhb: If they're all in the same timezone, that makes things quite a bit easier. You'll probably ignore leap seconds, too, I'm not sure unix time has leap seconds anyway, but...
<Arahael> eyJhb: The next decision is: Do you have to deal with daylight savings?
<gchristensen> you don't need to ignore leap seconds, you can use the built-in functions
<eyJhb> But that is also why it might be worth to actually just perform X queries, because if I have e.g. a view of a singel date, then I would maybe have 30 minutes interval, which would add up to ... 48 queries, which while I write this still suck
<eyJhb> gchristensen: NOT well :p
<gchristensen> hm?
<gchristensen> ah
<eyJhb> from_unixtime does not perform well :p That is what I am currently using :D
<eyJhb> arahael: ignore it as well
<gchristensen> you have 18.000.000 records which are interesting, ie: within the month you care about?
<eyJhb> It is somewhat fine, when we just look at a month. But if I e.g. wanted to do a recap of the year, and show for each month
<eyJhb> Then I would get a bunch of records still
<gchristensen> ah
<eyJhb> Then I would still get a bunch of records**
<Arahael> eyJhb: If you're going to be that sloppy, then I'd probably suggest making it even simpler and just doing it all in UTC - ie, zulu time.
<Arahael> eyeAnd yeah, using gchristensen's suggested function.
<Arahael> eyJhb: And yeah, using gchristensen's suggested function.
<eyJhb> I just don't like how slow it is.. A query takes 10-15 secounds
<gchristensen> eyJhb: out of curiosity, is the timestamp indexed?
<eyJhb> 1-2 secs are fine, but that just seems absurd
<gchristensen> have you EXPLAIN'd your query?
<gchristensen> (or EXPLAIN EXTENDED)
<eyJhb> Nope, I haven't yet, might make sense
<eyJhb> But in all honesty, never worked with that before
<eyJhb> Trying to see if timestamp is indexed
<gchristensen> if it is slow, and you want to make it fast, EXPLAIN and EXPLAIN EXTENDED are your first tools
<Arahael> Might even want to make a view that has the date column added to it, after you figure out that performance issue.
<eyJhb> Doesn't look like it
<eyJhb> That is what I considered doing, having a date column added
<eyJhb> And then just have a simple, on update timemodfified do X
<eyJhb> That would sovle most things I assume, but should betested
<AstraAdria4Ari> Hello everyone,
<AstraAdria4Ari> mint, popos, pureos, suse, clearlinux intel, gentoo still testing arch. while testing silverblue fedora I read about nixos, tried it yesterday and I am impressed by the uniqie packaage managment system, back on Sunday at work for more NixOS time (silverblue fedora OS+Nix pkg manager with DE:i3 or bspwm) true power of Linux and FOSS, the freedom and option to choose. Ari.
<AstraAdria4Ari> I am Alex a.k.a Ari, live in Europe work as researcher and do a lot of coding in Data Science (AI and MK) and "experimenting" with operating systems, mostly RedHat OS (left MS Win 2009, one year after it mac, since 2010 full Linux). As core OS for work I use always (no plan for a change for now) Fedora / Red Hat / CentOS or Scientific Linux, but for testing and experimenting I have tried a lot: debian, ubuntu, linux
<gchristensen> welcome, AstraAdria4Ari :)
<Arahael> eyJhb: Sounds too complicated, I was suggesting that purely as a means of making hte query itself simpler - the underyling table doesn't need to be modified, I suspect, except for perhaps adding an index.
<gchristensen> eyJhb: don't guess, your performance problems may be trivial to fix. use your tools to find out why it is slow
<aanderse> you know what makes you appreciate nixos? when you're carefully documenting (as you go) modifying a dozen different files under /etc and you realize even with commenting there is no easy way to tell exactly what the difference from upstream is, how steps must be followed in exact order sometimes, and how painful this is going to be to reproduce :\
<gchristensen> aanderse +100
<aanderse> setting up a bunch of pam stuff on ubuntu right now and i wish it was either on nixos, or i didn't know about nixos so i would be blissfully ignorant of how painful this is :P
<AstraAdria4Ari> Hello, joined discourse nixos org but I assumed IRC channel will be more active
<aanderse> AstraAdria4Ari: welcome
<eyJhb> arahael: but how would indexing help if I modify it with date etc.? - Of course gchristensen , I wouldn't just be shooting blind. Then I would be killed at the exam
<Arahael> eyJhb: That's the thing, it doesn't sound like you need to *modify* it with the date. You just want to group by date, right?
<AstraAdria4Ari> do you have any specific native build apps for data mining or ML/AI ?
<Arahael> eyJhb: You generally want to avoid changing the data.
<gchristensen> AstraAdria4Ari: a good channel for questions in #nixos, which is "about" nixos. -chat is barely about NixOS :)
<eyJhb> Yeah, basically :) And yes, I want to avoid that
<eyJhb> But I might have to investigate a little, because it seems like my own MySQL in Docker can execute it within 0.5 secs, which is GREAT, but my remote server takes 10-12 secunds, but I just can't imagine the CPU being that bad
<Arahael> eyJhb: It could be CPU, disk, or locking - you need to profile it, basically. I'm a postgresql guy (hobbiest), but I'm sure mysql will have tools that help here. (The above-mentioned EXPLAIN ?)
<eyJhb> With my very bad query, that works
<eyJhb> Yeah, I am guessing VM vs. bare metal
<Arahael> VM's today are very good, you should be nearly 100% bare metal performance.
<Arahael> I saw the PM... Ugh, mysql looks horrible - best ask a mysql guru how to interpret that (or check their docs)
<Arahael> eyJhb: Your remote server could easily have different indexes and different backends configured.
<Arahael> eyJhb: What could also be confounding things is the access pattern and your concurrency model - eg, if someone is writing to the table, you might not be able to read from it at the same time depending on how the database is setup.
<eyJhb> arahael: yeah, you might be right, but lets see :p One obstacle at a time!
<eyJhb> Hoping gchristensen has insight :D
<Arahael> eyJhb: DM'ing, are you two? :) Hope it goes well! OTherwise, you could ask #mysql or similar, I guess. But the first thing is to get the database to tell you why it's so slow.
<eyJhb> Yeah, it might have done some magic on the backend because I used it so much, but I am nore sure what. As I am not allowed to run EXPLAIN EXTENDED on my own LP
<eyJhb> And yeeees :D
* gchristensen is reminded he's not great at optimising SQL
<Arahael> It's easier when you have good tools.
<eyJhb> And I am reminded I hate it
<eyJhb> *purges table
<eyJhb> Optimised!
drakonis_ has joined #nixos-chat
<eyJhb> gchristensen: do you still have a mac?
<gchristensen> I do have various macs
<gchristensen> but I try not to run macos
drakonis1 has joined #nixos-chat
<eyJhb> So not joined the Thinkpad race?
<gchristensen> huh?
<eyJhb> Lenovo Thinkpad :p
drakonis2 has joined #nixos-chat
<gchristensen> the computer I touch and use is a dell XPS. I have a couple macs because I'm a contractor and clients like me to have their mac for some reason (whether or not I use it)
drakonis has quit [Ping timeout: 264 seconds]
<__monty__> They just throw macs at you? No wonder you operate the mac os build farm : >
<gchristensen> I wish I could :P
<Arahael> Which reminds me, does nix work nicely on the new macos verison - Catalina?
<__monty__> Hmm, isn't that the version that prohibited writing to /nix?
drakonis_ has quit [Ping timeout: 264 seconds]
drakonis1 has quit [Ping timeout: 246 seconds]
<Arahael> __monty__: Yes.
<Arahael> I think there's been some alternative found, though.
<__monty__> Seems so. #gladI'mstuckonoldOSXtho : >
__monty__ has quit [Quit: leaving]
drakonis has joined #nixos-chat
drakonis2 has quit [Ping timeout: 252 seconds]
cransom has joined #nixos-chat
johanot has quit [Quit: WeeChat 2.4]
drakonis_ has joined #nixos-chat
evanjs has quit [Quit: ZNC 1.7.4 - https://znc.in]
evanjs has joined #nixos-chat
drakonis has quit [Ping timeout: 276 seconds]
pie_ has joined #nixos-chat
waleee-cl has joined #nixos-chat
<eyJhb> gchristensen and arahael I think I might now the problem, I think it is building the indexes for users etc.
<eyJhb> Because this `SELECT q.id, qa.state, "2019-10-08" as period, COUNT(qa.quiz) AS participant_count FROM mdl_quiz as q LEFT JOIN mdl_quiz_attempts AS qa ON q.id = qa.quiz LEFT JOIN mdl_user AS u ON qa.userid = u.id WHERE u.institution = "REMOVED";` takes forever as well
<adisbladis> Hunk #1 succeeded at 245 (offset 3 lines).
<adisbladis> Oh please stop it patch, you are making me blush
<gchristensen> d'awww
drakonis_ has quit [Ping timeout: 276 seconds]
endformationage has joined #nixos-chat
drakonis has joined #nixos-chat
Jackneill has quit [Remote host closed the connection]
psyanticy has quit [Quit: Connection closed for inactivity]
Jackneill has joined #nixos-chat
drakonis has quit [Ping timeout: 265 seconds]
drakonis has joined #nixos-chat
Jackneill has quit [Remote host closed the connection]
drakonis has quit [Ping timeout: 276 seconds]
drakonis has joined #nixos-chat