Category Archives: Tech

Technical stuff. Not for normal humans.

Auto-indent + Paste = Murder

I use Vim pretty much exclusively as my IDE for all programming tasks be it Python, HTML, JavaScript… whatever. One of the Nifty Features™ of vim is auto-indentation which saves a lot of extra tab keystrokes in the longrun. The side effect of this Nifty Feature™ is that when you go to paste code from another source into a file you’re editing, it goes berserk and indents everything you just pasted. If the code you’re pasting is already indented, you end up with nested double indentation, and with languages like Python where whitespace is significant this can be a huge pain in the ass.

So I finally had enough and decided to share this solution along. This is using Vim’s pastetoggle, which allows you to map a keystroke to toggle auto-indentation when you want to paste stuff. Put this into your .vimrc:

To use it, you hit F3 to toggle paste mode, paste what you need to paste, then hit F3 again to turn it back off. And done!

Loki: A Python Infrastructure Protocol Suite

It’s been a while since I last posted, so I’m going back in time to write briefly about an awesome talk I attended at Black Hat 2010 in Las Vegas (last year, duh).

The toolkit is (poorly) named Loki. There are so many packages out there named Loki already, but I get where they were going with it.

It was written by three brilliant German fellows: Daniel Mende, Rene Graf and Enno Rey of ERNW. The primary focus is manipulating and exploiting infrastructure (layer 2/3) protocols, but of course these are all based on the entire protocol stack, so they have laid a nice framework that is way easier to use than other packet manipulation modules out there like Scapy, albeit more specialized. As of right now it has support for BGP, OSPF, ARP, ISIS, RIP, HSRP (v1 and v2), VRRP (v2 and v3), BFD, LDP, and MPLS. And it can spoof all of them!

Loki itself is pure Python, although it does have some system level dependencies. It centers mostly around loki.py, which is a GUI application that is its bread and butter. This app is effectively a router emulator that allows you to establish peerings and execute brute force attacks. For example all you need is a single BGP packet from the wire and you can then perform your brute-force agains the authentication key offline.

In some cases for protocols with no authentication such as VRRP–which is commonly used for providing redundant default gateways for Internet egress–you can push out another peer as a default gateway and take over that role with mind-blowing ease.

If you poke around you’ll see that they’ve encapsulated each protocol into its own module making almost of all of it reusable outside of the GUI. Additionally if you go to the main page of their site and scroll down to Tools, you’ll see a few other packages they’ve released.

For a more technical read, check out a recent blog post by Mike Poor.

Intro to Twisted

I presented a short Intro to Twisted at the SoCal Piggies meetup in Santa Monica last night. It went pretty well and I think I did a good job explaining the basics. I know it took me a long time to even become even remotely familiar with the way Twisted operates, so I felt like this was a good choice for a five-minute presentation. Short, sweet, and to the point!

Here is is for your viewing pleasure:

I borrowed the image on slide five from the Twisted Intro, which is a very excellent introductory tutorial written by Dave Peticolas. If you are curious about Twisted, what it can do, or are even experienced and want a great refresher, it’s definitely worth your time to give it a look!

On Using eval() in Python

Originally posted on StackOverflow.

I have used eval() in the past (and still do from time-to-time) for massaging data during quick and dirty operations. It is part of the toolkit that can be used for getting a job done, but should NEVER be used for anything you plan to use in production such as any command-line tools or scripts, because of all the reasons mentioned in the other answers.

You cannot trust your users–ever–to do the right thing. In most cases they will, but you have to expect them to do all of the things you never thought of and find all of the bugs you never expected. This is precisely where eval() goes from being a tool to a liability.

A perfect example of this would be using Django, when constructing a QuerySet. The parameters passed to a query accepts keyword arguments, that look something like this:

If you’re programmatically assigning arguments, you might think to do something like this:

But there is always a better way that doesn’t use eval(), which is passing a dictionary by reference:

By doing it this way, it’s not only faster performance-wise, but also safer and more Pythonic.

Moral of the story?

Use of eval() is ok for small tasks, tests, and truly temporary things, but bad for permanent usage because there is almost certainly always a better way to do it!

Logical Meltdown

I have been totally unable to accomplish any coding this week.  I’ve been rewriting this application at work and while I’ve made some decent progress, I haven’t gotten a damn thing done this week and it’s frustrating as hell!

Of course now that it’s Friday and I’m feeling the urge to start writing more code having to do with dongs.  What the fuck is wrong with me?  Don’t answer that.

I figured that by explaining some of what I’m struggling with, some magic will happen.  I need to blog more and work on my “explaining complex shit in a meaningful way” skills, so if this in any way interests you please read on.

WARNING: TECHNICAL SHIT BELOW!

The application I am rewriting is a MySQL database-driven app that has two interfaces:

  1. A command-line interface (CLI) utility written in Perl for direct manipulation (addition/removal of records) and querying (searching) of the database. Only engineers have access to this tool.
  2. A web application written in PHP for searching the database for non-engineers.  Mostly support and operations staff use this. This also provides a hacked up “API” that returns a list of comma-separated values (CSV) of matching records.
It’s something I wrote years ago using PHP & MySQL that is way more complex than I remember it and double-dog dare you more complex than it needed to be. It’s strange going back in time with my code, especially PHP (shame), and realizing how competent I was even then, but how unnecessarily convoluted I made some things.

There are a shit-ton of logical pitfalls in that code that have really thrown me off.  Being that we have a Perl CLI tool and a PHP web app, the much of the code is duplicated for each language. The tool was originally written in Perl with the web app piece tacked on as an afterthought.  At the time I was really into PHP’s native support for MySQL but was pretty much a novice so ended up making some downright lazy and bad choices.

With PHP and it’s bloated cousin Perl, there are so many ways to do the same thing that there really isn’t one right way to do things.  Typically that means there are that many more wrong ways to do them.  For example, horrible crap like this where we have PHP making a system() call to execute a Perl script from within index.php:

Couple that with my prior lack of regimented programming style because I originally learned to code from the back of a Cracker Jack box (aka the internet), and what I am working with is pretty much a nightmare.

“Why is that?”, you ask?  Well let me tell you!

First things first, the primary users of the app (aka the stakeholders) have been requesting some new features for a long time and with everything else going on it hasn’t really been a high priority for me.  Especially because it was written in Perl, which I have grown to hate passionately since I started using Python.

Next, the systems from which the CLI tool runs are being replaced, which means the app has to be moved to the new systems. We run a Red Hat offshoot distribution of Linux, so we use RPMs to manage what gets installed on machines.  What this means is that any Perl module dependencies need to be installed via RPM. With RPM and Perl there are always dependencies of your dependencies, and you can bet that some of those won’t be available in the repository. This opens a massive can of worms of having to download the module source and convert it to RPM.  For those of you who are familiar with Perl’s CPAN and RPM, you’re probably thinking “Just use cpan2prm!”. Right, if only it were that simple…

You see, the new hosts are running a 64-bit kernel, which means that the subset of available packages is even smaller because packages have to be specifically built (or rebuilt) for 64-bit installation. Even if it’s the same files inside of the package, it still has to be re-packaged for 64-bit.

So, I was left with the choice of either uprooting the tool from its current location, wrangling package dependencies, rebuilding packages, modifying the Perl code to shoehorn it into the new systems all the while overlooking the implementation of the requested features; or rewriting it using Python & Django.

Naturally, I chose to rewrite it because the thought of having to trudge through antiquated Perl hackery gave me diarrhea and I really don’t want to ever touch Perl or PHP again if I can help it. By rewriting the app in Python, I can implement the new features and bring the code into the present at the same time without having to go through bouts contemplating suicide. Annnddd… That leads us back to the nightmare that is Perl and PHP.

As I mentioned above, with Perl & PHP there are so many ways to do things that most of them are convoluted, confusing, hard-to-read, and just plain fucking wrong. With Python–and Django for that matter–there is usually only one right way to do things, and that way is pretty much always elegant.  Plus, one of the most beautiful things about Django is that it wholly and completely abstracts the database interaction from the source code.  That means not having to write complex SQL statements manually, and the less SQL I have to write, the better.

In meeting with the stakeholders on rewriting the application, we decided that all of the energy should be invested in replacing the web app with a real API with a RESTful interface. If you don’t know the basic premise of REST (which is a methodology, not a protocol), it’s that URLs should be easy-to-read, self-describing and beautiful. Beautiful URLs? WTF? How can a URL be beautiful?  I’m glad you asked!

The old PHP version of the API is very heavy on the query string, requiring GET variables to be passed in the URL itself which is not only cumbersome, but can get confusing and hard to read very quickly:

http://server.whatever.com/api.php?show=all&user=jathan

Compare that to a RESTful version of the same:

http://server.whatever.com/api/show/all/jathan/

This is where Django excels because it encourages pretty URLs by making it easier to use pretty URLs than not to.

So, cutting this post off at the knees, I’ve been pounding on this for the past few weeks, learning a lot about how NOT to do things from my old code and how TO do things using Django as my guide.  I made a lot of progress, got the API working, and a basic search interface that returns stuff all using the same code.

But this week, I’ve been stuck.  I haven’t made any progress, have been getting really confused, and just generally bashing my skull into the keyboard.  I think I’m just fried from trudging through all of Perl’s curly braces, semicolons, ampersands, dollar signs… OMG THE SYNTAX!  KILL ME NOW!!

Besides, it’s Fried-day anyways, so I hope that just by spelling this all out here that I’ll be able to dig back into the code after the weekend with a renewed perspective.

p.s. Fuck Perl.

p.p.s. Fuck PHP too, but wrap that rascal. She gets around!

More Dongs on a Friday. Surprise!!

Due to popular demand, mkdong has been modularized!!  I present to you dong.py version 0.0.1.

Enjoy:

There’s really no excuse. I should be ashamed of myself but I’m not.

Using Caps Lock on Your iPhone

I’ve had an iPhone for over a year and still hadn’t learned how to use the mysterious caps lock feature.  I enabled it in the settings, but had no idea how the hell to make use of it!  Of course it turns out to be just as intuitive as everything else on the iPhone. I just figured this out today so I thought I would share it.

Step 1:  Enable Caps Lock

Navigate to Settings > General > Keyboard and switch “Enable Caps Lock” to “ON”.

Enable Caps Lock

Enable Caps Lock

Step 2: Use Caps Lock

To use caps lock, all you have to do is double tap the shift key on the lower left of your keyboard.The key will turn blue indicating that you have successfully engaged CAPS LOCK.

CAPS LOCK ENGAGED

CAPS LOCK ENGAGED

To turn it off, just tap the shift key once more.

Your life has now been enriched, and I can move on with mine.

Blue Dongs for a Friday Afternoon

Today I wrote an awesome program called mkdong that will make a dong of your desired length and print it to your terminal, like this:

That last one is impressive, isn’t it? Hmm… Yeah, it’s Friday. What do you want from me? I still got work done! Cool thing is if the dong is too big, well then it throws an error:

“What is the point of this?”, you might ask yourself. That’s a good question. I’ve been so busy with other shit lately that I’ve barely had time to code. I suppose I was itching to write something, anything… Dongs!!

It all started harmlessly enough with a silly AIM conversation with my coding buddy at work. We were talking about a bug, and well, read on and you’ll see. It regressed quickly.

jughead: well there is the problem, now I just gotta figure out wtf jathanism: what did you do! jughead: nothing, it’s something in the rt stuff jathanism: hmm, maybe with a certain ticket jathanism: maybe one that is blank or something jathanism: or missing xml data jathanism: or might be a bug jughead: ok from now on jughead: bugs are not bugs jughead: they are dongs jathanism: ok jughead: “I found a dong in this code” jathanism: or might be a dong jughead: #1 deterrent of all linux exploits jughead: change root account to “poop”, chown -R poop.poop / jughead: no uid 0 jughead: no user root jathanism: what? jathanism: you rename uid 0 to poop ? jughead: poop will get it’s own uid jughead: and gid jathanism: oh jathanism: would that work? jathanism: there aren’t any hard-coded things to uid 0? jughead: no idea, lets try it jathanism: yeah let’s try it on marduk! jughead: hahahahfdsaf jughead: dsfa jughead: hmm…. /dev/ would have to be rebuilt jughead: MKNOD jughead: hate that shit jughead: MAKEDEV jughead: FUCK YOU LINUX jathanism: mkdong jathanism: aww yeah jathanism: i just made mkdong jathanism: % ./mkdong 5 8====0 jathanism: % ./mkdong 15 8==============0 jathanism: % ./mkdong 25 8========================0 jathanism: % ./mkdong 41 warning: a 41″ dong is too big! cannot be longer than 40″! jughead: you should modify that jughead: ()/()\\\\\\\\\\\\\\D jathanism: hahahlk jughead: looks much better jathanism: ok! jathanism: % ./mkdong 40 ()/()\\\\\\\\\\\\\\\\\\\\D jathanism: aww yeah

So I took the stupidity and ran with it and mkdong was born!

The initial dongs were a little primitive and sickly looking. So I took his suggestion and improved their visual style. Here is how it turned out:

We laughed. We joked. We Tweeted. And then it regressed even further:

jughead: dude everyone loves mkdong jathanism: aww yeah jathanism: it needs easter eggs jughead: DUDE jughead: MAKE IT PRINT IN BLUE jathanism: ok! jughead: how exactly does one “suck a fuck” jathanism: the ascii coloring fucks up the length jughead: you can just put the blue at the beginning jughead: and at the end jughead: doesn’t have to be each char jathanism: it’s not jughead: is there a dong in the code? jathanism: /tmp/mkdong 5 jathanism: will just use forward slashes instad jathanism: released jughead: man mkdong is the best ever

A feature request! I had to make it print in blue! But to do that I had to replace all of the “\” that make up the dong itself, with “/” so as to not have the ANSI escape codes eat up the extra backslashes. (Backslashes are interpreted characters, duh.) I also had to replace the print statement with a system call to echo -e so that the colorization would be interpreted. This is high tech shit, man!!

And then I released it to the public. So there you have it. Here is the final release of mkdong 2.0 for your pleasure:

Use it well. And remember they aren’t bugs, they’re dongs! Squish? Gross.

Python List Comprehension for Dummies

So I code a lot. I code in Python a lot. You might say I love Python. I might say you’re right.

One of the most powerful things about Python is its ability to iterate over ANYTHING as if it were a list. Lists, tuples, dictionaries, and even strings can all be iterated quickly and elegantly. Python also introduces a concept known as list comprehension which allows you to do rather complex filtering of list contents within a single statement.

To illustrate how awesome and powerful list comprehension is, let’s start with a basic example that is NOT using it:

So, let’s assume that we want to identify all even numbers inside of my_list, and put them into a new list called evens the old-fashoned way:

Why the old-fashioned way sucks First things first, the empty list called evens had to be declared ahead of time. This is because when we looped thru the list called my_list using the for statement, when the if test is performed on each item we have to reference evens by name to append() the even numbers to it.

Why list comprehension rocks With list comprehension, the logic that isolates the even numbers and the declaration of the list that will capture this output are compressed into a single statement:

The logic is encapsulated in [square brackets] indicating that the output will be a list. The list comprehension itself is the logic between the brackets that determines what will be in the list that it spits out.

So list comprehensions at their most basic level allow for compression of code and streamlining of logical statements. Advanced usage of list comprehension can get pretty silly, but then so can nested loop statements.  It supports nesting as many statements as you can throw at it so longs as they are syntactically correct.

If you find yourself coding shit like this:

Then maybe list comprehension is for you:

No offense to anyone named Joe, Jim, or Jon.

Proxying SSH with SOCKS (HTTP was so 2007)

By writing this I am assuming you know what SOCKS is, and you know what SSH is. If you don’t, here is a picture of a monkey fucking a coconut to make this visit worth your while:

Could be a melon, but looks like a co-co-nut.

Could be a melon, but looks like a co-co-nut.

So, there comes a time in a man’s life when people at work on the inside network need to access things on the internet.  This is called “proxying”.  Yes, yes, I know; very fascinating.  These secure machines on the inside network don’t have access to the internet by design (See RFC 1918).  It’s the most basic layer of obfuscation (a 25 cent word we use a lot in the security world) and protection from bad internet traffic, not including firewalls and all that other exciting stuff.

Ok so we want to let our secure hosts on the inside proxy SSH to the internet via our SOCKS server.

Assumptions:

  • A Unix/Linux machine with the latest version of netcat installed (assumed to be found at /usr/bin/nc).  All modern operating systems have this.  Stop whining.
  • A SOCKS proxy listening on TCP port 1080.
  • A remote internet server listening for SSH connections on TCP port 22.
  • You know what ~ means.  (Hint:  It’s shorthand for your home directory.)
Do the damn thing:

Create an entry in ~/.ssh/config. If this file doesn’t exist, create it. If it does, add this shit to the bottom:

Write, quit, and then test that shit! I am hoping that you gathered “proxythatshit” is the nickname we’re assigning this proxied connection to internet.com. By putting this stuff in the config file, it makes it easy to reuse.

Did you see that? It worked!! OMGZ!!JLk

A little breakdown:

ProxyCommand /usr/bin/nc -X 5 -x proxy.whatever.com:1080 internet.com 22

  • ProxyCommand: An OpenSSH directive that tells SSH how to proxy the connection
  • /usr/bin/nc: The path to the netcat binary and the ProxyCommand in question here.  Proxying is one of the many things netcat does.
  • -X 5: Tells netcat to use SOCKS version 5
  • -x proxy.whatever.com:1080: Tells netcat to proxy the connection using proxy.whatever.com on port 1080
  • internet.com 22: The name and port of the destination we’re trying to get to by way of the proxy
Why SOCKS?

You may be asking yourself, “Why not just use an HTTP proxy?”  Because HTTP proxies tend to be very picky about allowing you to proxy non-HTTP connections to destination ports other than the one you connected to.  In other words, if the proxy is listening on port 8080, good luck proxying a connection that isn’t HTTP (such as this SSH proxy thing) on anything other than port 80, 443, or 8080 it probably won’t work. If you’re using mod_proxy, it absolutely will not work.  Don’t ask me why.  It just doesn’t.  Squid might work, but it is a pain in the ass to setup.

There you have it.  Don’t blame me if you get fired because you were looking at a picture of a monkey fucking a coconut for the 52 seconds it took to read this.