Tag Archives: tutorial

Intro to Twisted

I presented a short Intro to Twisted at the SoCal Piggies meetup in Santa Monica last night. It went pretty well and I think I did a good job explaining the basics. I know it took me a long time to even become even remotely familiar with the way Twisted operates, so I felt like this was a good choice for a five-minute presentation. Short, sweet, and to the point!

Here is is for your viewing pleasure:

I borrowed the image on slide five from the Twisted Intro, which is a very excellent introductory tutorial written by Dave Peticolas. If you are curious about Twisted, what it can do, or are even experienced and want a great refresher, it’s definitely worth your time to give it a look!

On Using eval() in Python

Originally posted on StackOverflow.

I have used eval() in the past (and still do from time-to-time) for massaging data during quick and dirty operations. It is part of the toolkit that can be used for getting a job done, but should NEVER be used for anything you plan to use in production such as any command-line tools or scripts, because of all the reasons mentioned in the other answers.

You cannot trust your users–ever–to do the right thing. In most cases they will, but you have to expect them to do all of the things you never thought of and find all of the bugs you never expected. This is precisely where eval() goes from being a tool to a liability.

A perfect example of this would be using Django, when constructing a QuerySet. The parameters passed to a query accepts keyword arguments, that look something like this:

If you’re programmatically assigning arguments, you might think to do something like this:

But there is always a better way that doesn’t use eval(), which is passing a dictionary by reference:

By doing it this way, it’s not only faster performance-wise, but also safer and more Pythonic.

Moral of the story?

Use of eval() is ok for small tasks, tests, and truly temporary things, but bad for permanent usage because there is almost certainly always a better way to do it!

Dynamically Determining the Variables for a Django Template

This is something that came up on StackOverflow, and I took the time to provide a very detailed answer. I haven’t posted in a while, and since I spent time on this answer, here it is verbatim. Enjoy!

Original Question

I’d like to be able to instantiate a template from a file (presumably using the django.template.loader.get_template(filename) ), and then determine the set of variables that should be defined in whatever context it is passed.

(snip)

I tried synack’s answer and found that (with the replacement of filterexpression.token by filterexpression.var, to get the actual name of the variable without tags and so on) it returned the variables that are defined locally in the template, but did not work for variables that are defined in the parent that it extends.

So for example, suppose I have templates in two files:

toyparent.html:

{%block base_results%}
Django is {{adjective}}
{%endblock base_results%}

toychild.html:

{% extends "toyparent.html" %}

{%block base_results%}
{{block.super}}
I {{verb}} it.
{%endblock base_results %}

And I load the child template:

>>> toy=django.template.loader.get_template('toychild.html')

This renders properly:

>>> toy.render(django.template.Context(dict(adjective='cool',verb='heart')))
u'\n    \nDjango is cool\n\n    I heart it.\n\n'

But I can’t get the two variables from it:

>>> v=toy.nodelist.get_nodes_by_type(VariableNode)
>>> for k in v: print k.filter_expression.var
...
block.super
verb

My Answer

You are able to visually inspect a template and observe the presence of any “Variable Node” objects in that template’s nodelist:

>>> from django.template import Template, Context
>>> t = Template("Django is {{ adjective }} and I {{ verb }} it.")
>>> t.nodelist
[<Text Node: 'Django is '>, <Variable Node: adjective>, <Text Node: ' and I '>, <Variable Node: verb>, <Text Node: ' it.'>]

These are of the type VariableNode, which is a class that can be directly imported for use in comparisons. Any Node instance has a getnodesby_type() method that can be called against a nodelist, which return all nodes of that type for the template. Example:

>>> from django.template import VariableNode
>>> varnodes = t.nodelist.get_nodes_by_type(VariableNode)
>>> varnodes
[<Variable Node: adjective>, <Variable Node: verb>]

So now you have a list of the variables for the template. This will need to be taken a step further to extract the actual name of each variable without peforming stupid string-slicing tricks on their repr names.

The variable name itself is stored in filter_expression.token for each VariableNode:

>>> varnodes[0].filter_expression.token
u'adjective'

And so a simple list comprehension gets us all of the variable names for the template:

>>> template_vars = [x.filter_expression.token for x in varnodes]
>>> template_vars
[u'adjective', u'verb']

So, not the simplest solution, but if there is a better way I don’t know about it.

Bonus: A function!!

from django.template import VariableNode
def get_template_vars(t):
   varnodes = t.nodelist.get_nodes_by_type(VariableNode)
   return [x.filter_expression.token for x in varnodes]

Ok, it’s not so complex after all!

Follow-up Edit: Getting variables from parent templates

(This follow-up is using the information from the updated question).

This is where it does actually get complex because the nodelist of the toy template is a single ExtendsNode (in this case).

>>> toy.nodelist
[<ExtendsNode: extends "mysite/toyparent.html">]

I would imagine that in larger templates there could be multiple ExtendsNode objects. Anyhow, if you inspect the ExtendsNode, and extract the parent template from it, you are able to treat the parent the same as my original example:

>>> enode = toy.nodelist[0]
>>> enode.parent_name
u'mysite/toyparent.html'
>>> parent = enode.get_parent(enode.parent_name)
>>> parent
<django.template.Template object at 0x101c43790>
>>> parent.nodelist.get_nodes_by_type(VariableNode)
[<Variable Node: adjective>]

And there is your adjective variable extracted from the parent template. To perform a test against an ExtendsNode you can import the class from django.template.loader_tags:

>>> from django.template.loader_tags import ExtendsNode
>>> ext = toy.nodelist.get_nodes_by_type(ExtendsNode)
>>> ext
[<ExtendsNode: extends "mysite/toyparent.html">]

So, you could do some tests against templates for the presence of an ExtendsNode and walk backwards to the parent template and individually get those variable names. However, this is starting to seem like a can of worms.

For example, if you were to do this:

>>> toy.nodelist.get_nodes_by_type((ExtendsNode, VariableNode))
[<ExtendsNode: extends "mysite/toyparent.html">, <Variable Node: block.super>, <Variable Node: verb>]

Now you’ve got the ExtendsNode and VariableNode objects and it just starts to get confusing. What do we do then? Do we attempt to ignore any block variables returned from such tests? I don’t know!!

In any case, this is the information you wanted, but I don’t think that this is a practical solution. I insist that there is still probably a better way. It might be worth looking into what you are trying to solve and see if there is another approach you can take.

Python List Comprehension for Dummies

So I code a lot. I code in Python a lot. You might say I love Python. I might say you’re right.

One of the most powerful things about Python is its ability to iterate over ANYTHING as if it were a list. Lists, tuples, dictionaries, and even strings can all be iterated quickly and elegantly. Python also introduces a concept known as list comprehension which allows you to do rather complex filtering of list contents within a single statement.

To illustrate how awesome and powerful list comprehension is, let’s start with a basic example that is NOT using it:

So, let’s assume that we want to identify all even numbers inside of my_list, and put them into a new list called evens the old-fashoned way:

Why the old-fashioned way sucks First things first, the empty list called evens had to be declared ahead of time. This is because when we looped thru the list called my_list using the for statement, when the if test is performed on each item we have to reference evens by name to append() the even numbers to it.

Why list comprehension rocks With list comprehension, the logic that isolates the even numbers and the declaration of the list that will capture this output are compressed into a single statement:

The logic is encapsulated in [square brackets] indicating that the output will be a list. The list comprehension itself is the logic between the brackets that determines what will be in the list that it spits out.

So list comprehensions at their most basic level allow for compression of code and streamlining of logical statements. Advanced usage of list comprehension can get pretty silly, but then so can nested loop statements.  It supports nesting as many statements as you can throw at it so longs as they are syntactically correct.

If you find yourself coding shit like this:

Then maybe list comprehension is for you:

No offense to anyone named Joe, Jim, or Jon.

Proxying SSH with SOCKS (HTTP was so 2007)

By writing this I am assuming you know what SOCKS is, and you know what SSH is. If you don’t, here is a picture of a monkey fucking a coconut to make this visit worth your while:

Could be a melon, but looks like a co-co-nut.

Could be a melon, but looks like a co-co-nut.

So, there comes a time in a man’s life when people at work on the inside network need to access things on the internet.  This is called “proxying”.  Yes, yes, I know; very fascinating.  These secure machines on the inside network don’t have access to the internet by design (See RFC 1918).  It’s the most basic layer of obfuscation (a 25 cent word we use a lot in the security world) and protection from bad internet traffic, not including firewalls and all that other exciting stuff.

Ok so we want to let our secure hosts on the inside proxy SSH to the internet via our SOCKS server.

Assumptions:

  • A Unix/Linux machine with the latest version of netcat installed (assumed to be found at /usr/bin/nc).  All modern operating systems have this.  Stop whining.
  • A SOCKS proxy listening on TCP port 1080.
  • A remote internet server listening for SSH connections on TCP port 22.
  • You know what ~ means.  (Hint:  It’s shorthand for your home directory.)
Do the damn thing:

Create an entry in ~/.ssh/config. If this file doesn’t exist, create it. If it does, add this shit to the bottom:

Write, quit, and then test that shit! I am hoping that you gathered “proxythatshit” is the nickname we’re assigning this proxied connection to internet.com. By putting this stuff in the config file, it makes it easy to reuse.

Did you see that? It worked!! OMGZ!!JLk

A little breakdown:

ProxyCommand /usr/bin/nc -X 5 -x proxy.whatever.com:1080 internet.com 22

  • ProxyCommand: An OpenSSH directive that tells SSH how to proxy the connection
  • /usr/bin/nc: The path to the netcat binary and the ProxyCommand in question here.  Proxying is one of the many things netcat does.
  • -X 5: Tells netcat to use SOCKS version 5
  • -x proxy.whatever.com:1080: Tells netcat to proxy the connection using proxy.whatever.com on port 1080
  • internet.com 22: The name and port of the destination we’re trying to get to by way of the proxy
Why SOCKS?

You may be asking yourself, “Why not just use an HTTP proxy?”  Because HTTP proxies tend to be very picky about allowing you to proxy non-HTTP connections to destination ports other than the one you connected to.  In other words, if the proxy is listening on port 8080, good luck proxying a connection that isn’t HTTP (such as this SSH proxy thing) on anything other than port 80, 443, or 8080 it probably won’t work. If you’re using mod_proxy, it absolutely will not work.  Don’t ask me why.  It just doesn’t.  Squid might work, but it is a pain in the ass to setup.

There you have it.  Don’t blame me if you get fired because you were looking at a picture of a monkey fucking a coconut for the 52 seconds it took to read this.

Creating read-only user accounts on ScreenOS

Need to create a read-only account on a NetScreen (ScreenOS 6.x or lower) firewall?

It’s simple:

And there you have it. Now let’s test it:

It works! Notice the limited command set available:

Now hop to it!