
The Magic CmdLine and how I got it back - ttsiodras
http://users.softlab.ece.ntua.gr/~ttsiod/bashheimer.html
======
kazinator
GDB can almost be a dynamic computing environment:

    
    
      $ gdb ./txr
      GNU gdb [ ... ]
      [ ... ]
      Reading symbols from /home/kaz/txr/txr...done.
      (gdb) b eval
      Breakpoint 1 at 0x808f6b0: file eval.c, line 969.
      (gdb) r -p '(+ 2 2)'
      Starting program: /home/kaz/txr/txr -p '(+ 2 2)'
    
      Breakpoint 1, eval (form=0xb7fb919c, env=0xb7fb92ac, ctx_form=0xb7fb919c)
          at eval.c:969
      969     {
      (gdb) p d(form)
      (+ 2 2)
      $1 = void
      (gdb) p d(cons(form, form))
      ((+ 2 2) + 2 2)
      $2 = void
      (gdb) p d(cdr(form))
      (2 2)
      $3 = void
      (gdb) p d(cons(0x1, 0x1))
      (0 . 0)
      $4 = void
      (gdb) p cons(0x1, 0x0)
      $5 = (obj_t *) 0xb7fb921c
      (gdb) p cons(0x9, $5)
      $6 = (obj_t *) 0xb7fb920c
      (gdb) p cons(0x19, $6)
      $7 = (obj_t *) 0xb7fb91fc
      (gdb) p d($7)
      (6 2 0)
    

You can test functions that are not even called anywhere from the C code. The
d function is such a function; it's only there for use out of GDB.

    
    
      (gdb) p (0)
      $10 = 0
      (gdb) p d(0)
      nil
      $11 = void
      (gdb) p t
      $12 = (val) 0xb7fe5eac
      (gdb) p d(t)
      t
      $13 = void
      (gdb) p typeof(t)
      $14 = (obj_t *) 0xb7fe5d3c
      (gdb) p d(typeof(t))
      sym
      $15 = void
      (gdb) p d(symbol_name(t))
      "t"
      $16 = void
      (gdb) p d(symbol_name(0))
      "nil"
      $17 = void
    

Take out the trash, and show it:

    
    
      (gdb) p gc()
      $18 = void
      (gdb) p d($7)
      #<garbage: 0xb7fb91fc>

~~~
zilog80
And in combination with conditional breakpoints executing commands, you can
have GDB "dance" any application to whatever tune you wish:

commands ... call d(cons(form)) call symbol_name(t) cont end

It really is a marvelous tool - unfortunately, people don't know anything
except the basics (if that)

~~~
voltagex_
Is there anywhere I can learn specifically about the commands you mentioned?

~~~
ttsiodras
I've used them too - just "info gdb" then search (/) for "Breakpoint Command".

------
sp332
Just curious about one of the last steps:

    
    
      cat /proc/53165/exe > /tmp/oldBash
    

I would have reached for 'cp /proc/53165/exe /tmp/oldBash', instead of using
cat. Is there an advantage to using cat here, or is it just the same?

~~~
ttsiodras
Indeed! No idea what went through my brain when I wrote that :-)

~~~
zokier
Could you just use `gdb --pid 53165 /proc/53165/exe`?

~~~
hobofan
If that works, is there any reason why gdb shouldn't automatically use the
/proc/PID/exe?

~~~
zb
Probably because /proc is very Linux-specific, and gdb is not.

------
andmarios
Why not ctrl+c, then arrow up and enter? If the command can't be stopped even
for a few milliseconds then there is something wrong with it.

~~~
ttsiodras
I didn't want to send any signal - because I could not remember what I had
done, and was afraid for sideeffects (I did remember that the logic had to do
with database actions).

I wanted to see the command without "messing" with the execution in any way -
after all, it was running fine for 6 months, why interrupt it (and face
potential effort in fixing messed up state) if I could see the command "from
the outside"?

And that I did :-)

~~~
__david__
C-z is actually your friend here. I've never ever seen SIGSTOP affect anything
negatively. I even SIGSTOP Mac OS X apps sometimes. They just beachball until
you SIGCONT them, then they happily continue, oblivious.

If you're worried about timing out of the process being unresponsive, just C-z
and then "bg" real quick. Then you can up arrow at your leisure and "fg" when
you've copied the command away...

~~~
ttsiodras
Try doing the Ctrl-z/fg sequence in this: "while true ; do sleep 1 ; done".
Under Linux at least, you'll see that after 'fg', the loop ends :-(

Apparently C-z followed by fg is not bulletproof...

~~~
chaosfox
Thats an interesting example.

zsh does the right thing, when you bg it back it recovers the loop as it was.
Bash seems to forget about the loop and only recovers the "sleep 1", when it
starts executing again it sleeps 1 and exits.

If you want to make sure it will work you should wrap it around another bash:

    
    
        bash -c 'while true ; do sleep 1 ; done'
    

should work fine.

~~~
__david__
Parens work too:

    
    
       (while true ; do sleep 1 ; done)

------
metafex
The tip about getting the binary from /proc is also helpful if you upgrade
screen/tmux and the newer version doesn't allow you to reattach to it. With
that you can use the old one to reattach, work your magic, and restart the
session.

~~~
zilog80
You just explained why I could not attach to an already existing screen last
month - it all fits now...

------
barrkel
When I read the problem description, I decided to try and solve it myself.

I used gcore to dump the running bash process, then ran strings on the core
dump, then grepped for likely command line stuff. 'while' is reasonable choice
for a loop. There aren't that many strings in the dump file in any case, most
of the sizeable ones are function definitions and environment variables.

Relying on gdb makes getting to the right data much more precise, but my
approach took about 20 seconds.

------
vacri
I saw things being saved to /tmp, and kept expecting a turn for the worse from
a sudden reboot...

Fantastic article though - I've learned a couple of things from it, thank you.

~~~
ttsiodras
My pleasure :-)

------
eponeponepon
Great story... but I read through it hoping to see the fabled command in its
entirety at the end! Any chance..? :)

~~~
ttsiodras
God, no :-)

Some people here are already rather mean in their comments - imagine what
would happen if I revealed The Magic CmdLine (TM)...

No, that privilege is reserved for the Inner Circle :-)

But I'll give you a teaser... It involved a lot of ImageMagick and "pdfimages"
and "zxing" and "tesseract" and "pdfjoin" invocations... and the uploaded
files were PDFs with scanned barcoded pages.

~~~
eponeponepon
Aw, there's nothing worse than a mystery! ;)

(I can imagine the gory details tbh - it sounds like there is every chance I
have had a _very_ similar command for a pretty similar purpose running for a
good long while now... except mine's in a VBScript..!)

~~~
ttsiodras
:-)

Across the road (on Reddit/programming), a commenter already described me
pretty well:

"Everything about your (rather interesting) story... reaffirms everything I
know about Linux sysadmins - if you encounter a problem, your go-to solution
is to write a script ... Honestly, you guys would script a sandwich if you
could :P"

The only thing wrong in that sentence, is that I am not a sysadmin (hence the
knowledge of the Dark Arts - of GDB :-)

------
krick
Now this is both interesting and well written post indeed.

------
claar
This was a fun read and great hack. Thanks for sharing.

~~~
ttsiodras
My pleasure! It was a lot of fun hunting all this down, I just had to share it
:-)

~~~
slowmovintarget
It is indeed cool, but the single best lesson to come of it might be to start
the work in a script in the first place. Still, cool beans. :)

------
eridal
the best I've read since a long time!!

------
MDCore
Nice hack! I hope you learn to start save your notes/commands/snippets to a
text file as you go.

~~~
ttsiodras
I do, indeed - and in Git repos, usually :-)

How that one got by me is a mystery - let's chalk it up to stress and pretend
it never happened :-)

~~~
MDCore
Hehe. And to contradict myself, if you HAD written it all down you wouldn't
have the cool story :)

------
jarin
Er, why not just use the `history` command (or hit the up arrow)?

~~~
ttsiodras
Good question. Well, when you have forgotten everything about a 10 lines long
command line you wrote 6 months ago, you don't know what side effects a signal
may cause... You don't have the heart to signal anything, except surrender.
Luckily, I found a workaround :-)

~~~
albertoleal
While I'm unable to come with this on my own, the article was pretty easy to
digest and understand.

It seems that the take-away lesson from the article is that you should save it
in a bash script.

------
ErikRogneby
screen doesn't have history?

~~~
keeperofdakeys
It has a finite buffer, so I imagine 6 months worth of logs would have easily
blown away the history of you running the command.

~~~
zilog80
You can configure it though, to be much larger than its default settings.
Still, 6 months... he'd probably have to hack it as he did, no matter what :-)

------
kelvin0
Cool tricks, but reads like a how-to of things not to do when writing any type
of code and deploying it.

A Jedi? Maybe at getting himself out of the quicksand he lays himself into ...

~~~
ttsiodras
Oh come on :-)

You're telling me that you've never - under tremendous pressure - hacked
things that quickly "handle" something - and then forgotten about them till
much later?

Jeez, tough crowd :-)

~~~
mst
Those who do, blog. Those who don't, snark at those blog posts on HN.

------
hammerandtongs
I appreciate the spirit and the knowledge that you wrote this blog post with
BUT -

The extra few seconds to put this in a .sh file in the first place ...
priceless. Even for the basic case of debugging it and getting it working.

I don't think in the 20-25 years I've been "doing" unix that I've ever done
this (I doubt anyone that has worked with me considers me ocd fwiw but this is
beyond the pale).

Behaviors like this lead to "magic" work environments and are just pants on
head places to work at/with.

I would be really mad the first time I found you doing this and I'd look for a
different work situation for one of us the second time :)

Thanks for the cool post.

------
bogomipz
you're a jedi because you discovered the /proc file system? Right, OK.

------
ricket
My mom has Alzheimer's disease. I either sympathize with your situation, or if
you were using it as an expression, I find the misuse a bit offensive.
Particularly in this context for me, it brings up a painful memory of one of
the first signs of Alzheimer's in my mom, when she called me because she had
entirely forgotten her computer password.

As a sidenote to anyone reading this, I really appreciate that the Hacker News
community occasionally posts and upvotes Alzheimer's articles. I read every
one that I see.

Anyway, I beg that you might not use "Alzheimer's" as an expression for
forgetfulness, just as you might avoid calling someone "ADD" when they
multitask to a fault.

~~~
mgraczyk
It's worth noting that the author appears to be in Greece. Assuming he's a
non-native speaker, it doesn't really make sense to be "offended" by his
misuse of the term.

Maybe a better way to put it would be "We don't typically use 'Alzheimer's' to
refer to forgetfulness because it brings up negative emotions for some
people."

~~~
oxioxi
Greek language is more about emotion than content. Hyperbole is used to
emphasize meaning. I cannot forget he first time I went to Greece and my aunt
kissed me, bit me, and said she was going to eat me. I was 6. I was petrified.
It is common in Greece to express disappointment affectionately by saying ' I
am going to kill you'. It is our cultural ignorance in the US that makes
Americans easy targets for ridicule. If we project ourselves as a superpower
to the world, then where indeed are our superpowers of understanding?

