Notes from 15 October, 2015: loop invariants, psql will eat your soul, bad server.log pickup lines, TV ideas

Autumn Bridge Term continues!

I have some outstanding questions from the end of chapter 1 of CLRS (outstanding questions…they’re terrific!) (ho ho ho), and I will ask about them later, but also I dove right into chapter 2.

Unsurprisingly, it gets denser pretty fast. I am reading and rereading and rerereading a bunch of things. I have a basic grasp on, say, loop invariants, but I absolutely do not yet grok them.


Had a part-social-catchup, part-lunch-with-Nathan-and-his-mom-who’s-in-town lunch, and have significantly decreased my FOMO related to RC chat (Zulip). (I made a lot of choices about what NOT to read, which I am really proud of, and chose to read a small number of threads. All hail good threading.)

Now I do not have a big glaring “6949 UNREAD MESSAGES” thing hovering over the icon, and also I haven’t been keeping Zulip open, which makes it even easier. And I didn’t have to read 6949 messages to get there. (Phew!)

Decided to bravely tackle a Postgres problem with Nathan after lunch. We’ve previously tried to work on programming problems before, and it turns out that that might be a dangerous thing to do with your partner?

This time went a lot smoother, though, than any previous attempt, which I suspect is largely due to practicing “I don’t know” and related skills at RC.

Let’s back up in the terminal and see what we discovered! My brain is exhausted, but I know that I can understand my past.

? ? ? ? ? ? ? psql
psql: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/tmp/.s.PGSQL.5432"?

As you may remember from previous adventures with my prompt, the “? ? ? ? ? ? ?” is literally my bash prompt and I refer to it by its shortname sometimes, “chickparty,” because that’s what you get with a baby chicken and six balloons.

Anyway.

All the googling in the world yesterday & today helped me find some interesting information: namely, be exquisitely careful with your postmaster.pid, because you can do DEEP DAMAGE if you go messing with it all willy-nilly, but sometimes you MUST mess with it. This was not one of those times, regrettably.

For an excellent explanation of one possible path here, see this StackOverflow post. I’m especially impressed that the OP  ultimately solved their own problem, and THEN took the time to retrace their steps, explaining each one for the benefit of others.

I tweaked the find command just in case I was missing something, but there was indeed no postgres…except, as Nathan explained to me, the instance of grep that was searching for postgres. Sigh. Now I know!

  501  6548  3487   0  3:19PM ttys016    0:00.00 grep postgres

Ugh. Go find yourself, grep.

Alas, the end of my server log had an entirely different error than the one in the SO post:

LOG:  skipping missing configuration file "/usr/local/var/postgres/postgresql.auto.conf"
FATAL:  database files are incompatible with server
DETAIL:  The data directory was initialized by PostgreSQL version 9.2, which is not compatible with this version 9.4.4.

Yesterday, I had PAGES upon pages upon pages upon pages of this, almost endlessly repeating (there was some other stuff way way way up high, and the only dates I found for THOSE were in early 2014…hmm!). I scrolled with the mouse, the scrollbar barely moved, and I ultimately just had to drag and drop it like some kind of…person who does inelegant things.

For fun, and because both the internet and Nathan had suggested this, I ran tail -f,  which follows (aha! there’s the f) the progress of the file until you tell it to stop. Sure enough, guess what kept adding to itself every few beats:

? ? ? ? ? ? ? tail -f /usr/local/var/postgres/server.log 
DETAIL:  The data directory was initialized by PostgreSQL version 9.2, which is not compatible with this version 9.4.4.
LOG:  skipping missing configuration file "/usr/local/var/postgres/postgresql.auto.conf"
FATAL:  database files are incompatible with server
DETAIL:  The data directory was initialized by PostgreSQL version 9.2, which is not compatible with this version 9.4.4.

(and on and on until control-c stopped it)

SPOILER ALERT it is the postgres logs! My friend, you have been so, so busy for SUCH a long time, huh? Your legs must be tired (why) because you have been running on my computer for all eternity (groan) ho ho ho

This postgres directory had all kinds of goodies in it:

? ? ? ? ? ? ? ls
PG_VERSION	pg_clog		pg_multixact	pg_subtrans	pg_xlog		postmaster.pid
base		pg_hba.conf	pg_notify	pg_tblspc	postgresql.conf	server.log
global		pg_ident.conf	pg_stat_tmp	pg_twophase	postmaster.opts

And yeah, doing cat PG_VERSION netted a clean 9.2. Rats. 9.2 is apparently five hundred kinds of trouble! Down with 9.2; long live 9.4 until the next Extremely Different New Version comes out.

We looked at disk usage here with the -h flag, which stands for human? It displays it not in…another format which is trickier.

? ? ? ? ? ? ? du -h
6.0M	./base/1
6.0M	./base/12265
6.0M	./base/12270
6.0M	./base/16384
 24M	./base
428K	./global
8.0K	./pg_clog
8.0K	./pg_multixact/members
8.0K	./pg_multixact/offsets
 16K	./pg_multixact
8.0K	./pg_notify
4.0K	./pg_stat_tmp
8.0K	./pg_subtrans
  0B	./pg_tblspc
  0B	./pg_twophase
 16M	./pg_xlog
 83M	.

All the base are belong to us directories correspond to previously existing databases (from a zillion years ago when I managed QA reports for support, and also possibly from an open source day at Grace Hopper a few years ago?). Which is neat! I did not know that. That is a thing I learned today, too.

Because I decided that I did not care about these old databases that I have not looked at in literally years, we sallied forth with a rm -rf *,  which felt so so spooky to run.

Just. Whole postgres folder there. GONE. Run ls, completely empty.

Running psql still gave the same error (about /tmp/.s.PGSQL.5432), BUT, I ran ls again in /usr/local/var/postgres/, and we NOW have…dun dun dun…a server.log file! It’s Not Nothing!

Look at the cool new stuff at the end of server.log:

postgres cannot access the server configuration file "/usr/local/var/postgres/postgresql.conf": No such file or directory

It’s different! And then it was just THAT forever and ever, ^C (which is, I don’t know, like a computer’s “amen”? sure).

Okay so maybe not the most thrilling, but it’s Progress, and that is what I was after today.

Tried to get something going, since we’d now cleaned out all the old misbehaving stuff. Ran a series of commands that did not impress the computer:

  • initdb? no.
  • initdb -D .? no, initdb: directory “.” exists but is not empty. If you want to create a new database system, either remove or empty the directory “.” or run initdb with an argument other than “.”.
  • How about rm server.log; initdb? nooope. How about…
  • rm server.log; initdb -D . ? JACKPOT. By which I mean “success messages.”

So about that empty postgres directory. What’s it got in it NOW?

? ? ? ? ? ? ? ls
PG_VERSION		pg_ident.conf		pg_snapshots		pg_xlog
base			pg_logical		pg_stat			postgresql.auto.conf
global			pg_multixact		pg_stat_tmp		postgresql.conf
pg_clog			pg_notify		pg_subtrans		postmaster.opts
pg_dynshmem		pg_replslot		pg_tblspc		postmaster.pid
pg_hba.conf		pg_serial		pg_twophase		server.log

It’s not nothing! [distant cheers] The crowd goes wild [some woo-girls scream “WOO” in the background] everything is coming up Chickparty

And look at this adorableness (this is the whole server.log, not just the constantly expanding tail of suffering):

? ? ? ? ? ? ? cat server.log 
LOG:  database system was shut down at 2015-10-15 15:31:42 PDT
LOG:  MultiXact member wraparound protections are now enabled
LOG:  autovacuum launcher started
LOG:  database system is ready to accept connections

So then we whacked a few sticks against the wall, eventually creating a database, but that part almost feels like it will be helpful to struggle through a few times.

psql
(no)

psql foo
(no)

psql postgres
(yeah, but, what if we want something else. let's try to do something else.)

psql postgres 'create database foo'
(no)

psql postgres -c 'create database foo'
CREATE DATABASE

(thus spake Zarapostgres)

This puzzled Nathan for a bit, too (we didn’t hit on the perfect command the first time, anyway), which made me feel way better, because he does Postgres-y things a bunch for work. It’s nice to have people model that expertise does not equal omniscience — part of being experienced is knowing how to be productive and curious when stuck, I think.

So! Now I still have four CSVs full of all numbers (one date, many many many floats with 15 digits past the decimal…if they’re already double precision types, I’m kinda wishing there were a non-CSV format for these SUCH AS perhaps a database file). But I also have a working postgres to wrestle with tomorrow.

Postgres wrestling! Coming to a television near you. It’s gonna be big.

Leave a Reply

Your email address will not be published. Required fields are marked *