Tuesday, December 20, 2011

POST with a list in django

In your template, you may want to do a post with a list of values, like this:
$.post('{% url affiliate.views.manage_access %}', {
    id_list: [1,2,3],
  });
You can't just grab the value from the post like you normally would, since it will only grab one of the items.  So this won't work:
def manage_access(request):
    id_list = request.POST['id_list']
You must instead do this:
def manage_access(request):
    id_list = request.POST.getlist('id_list[]')

Wednesday, August 10, 2011

Django template tag escaping

In django, template tags look like this for a variable:
{{ variable_name }}
Sometimes you want to print out "{{" or "}}" without django trying to interpret it. Here's an example of how to escape the the template tags:
{% templatetag openvariable %} blah {% templatetag closevariable %}

Wednesday, August 3, 2011

Quick regex match example in python

This is an example of matching the "blahblah" that is between "this" and the first "end" (i.e. it's not greedy).

>>> s = 'try to match thisblahblahend and not that end or else...'
>>> r = re.compile('this(?P.+?)end')
>>> m = r.search(s)
>>> m.group('my_match')
'blahblah'

Monday, August 1, 2011

Python: How to distinguish last loop of iteration

Sometimes you want to treat the last element in a list differently when you're iterating through. Here's how to do that:

prev = None
for x in range(10):
    if prev is not None:
        print "not last:", prev
    prev = x
print "last:", prev

Output:
not last: 0
not last: 1
not last: 2
not last: 3
not last: 4
not last: 5
not last: 6
not last: 7
not last: 8
last: 9

Python Padding with PKCS7

Here's the definition of PKCS7 padding (from RFC 2315):

RFC 2315, section 10.3, note #2:
     2.   Some content-encryption algorithms assume the
          input length is a multiple of k octets, where k > 1, and
          let the application define a method for handling inputs
          whose lengths are not a multiple of k octets. For such
          algorithms, the method shall be to pad the input at the
          trailing end with k - (l mod k) octets all having value k -
          (l mod k), where l is the length of the input. In other
          words, the input is padded at the trailing end with one of
          the following strings:

                   01 -- if l mod k = k-1
                  02 02 -- if l mod k = k-2
                              .
                              .
                              .
                k k ... k k -- if l mod k = 0

          The padding can be removed unambiguously since all input is
          padded and no padding string is a suffix of another. This
          padding method is well-defined if and only if k < 256;
          methods for larger k are an open issue for further study.

And here's how to implement it in python:

class PKCS7Encoder():
    """
    Technique for padding a string as defined in RFC 2315, section 10.3,
    note #2
    """
    class InvalidBlockSizeError(Exception):
        """Raised for invalid block sizes"""
        pass

    def __init__(self, block_size=16):
        if block_size < 2 or block_size > 255:
            raise PKCS7Encoder.InvalidBlockSizeError('The block size must be ' \
                    'between 2 and 255, inclusive')
        self.block_size = block_size

    def encode(self, text):
        text_length = len(text)
        amount_to_pad = self.block_size - (text_length % self.block_size)
        if amount_to_pad == 0:
            amount_to_pad = self.block_size
        pad = chr(amount_to_pad)
        return text + pad * amount_to_pad

    def decode(self, text):
        pad = ord(text[-1])
        return text[:-pad]


Example use:
>>> # basic use
>>> encoder = PKCS7Encoder()
>>> padded_value = encoder.encode('hi')
>>> padded_value
'hi\x0e\x0e\x0e\x0e\x0e\x0e\x0e\x0e\x0e\x0e\x0e\x0e\x0e\x0e'
>>> len(padded_value)
16
>>> encoder.decode(padded_value)
'hi'

>>> # empty string
>>> padded_value = encoder.encode('')
>>> padded_value
'\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10\x10'
>>> len(padded_value)
16
>>> encoder.decode(padded_value)
''

>>> # string that is longer than a single block
>>> padded_value = encoder.encode('this string is long enough to span blocks')
>>> padded_value
'this string is long enough to span blocks\x07\x07\x07\x07\x07\x07\x07'
>>> len(padded_value)
48
>>> len(padded_value) % 16
0
>>> encoder.decode(padded_value)
'this string is long enough to span blocks'

>>> # using the max block size
>>> encoder = PKCS7Encoder(255)
>>> padded_value = encoder.encode('hi')
>>> len(padded_value)
255
>>> encoder.decode(padded_value)
'hi'

Friday, July 15, 2011

In django, formsets (formset_factory) allow you to list out a bunch of forms one after the other.  This is useful when you want to allow the user to add/remove objects.  For example, if you have a recipe application where the user can create new recipes, you could have an ingredient formset that allows the user to add more ingredients to a recipe.  In this case, you could also use an inline formset (inlineformset_factory) that is linked with the recipe that you're adding the ingredients for.

In my case, I had a number of questions that I wanted the user to answer.  There were a finite set of questions, and I wanted each question/answer to be listed out on a separate line on the page.  For my models I have a Question table and an Answer table.  The answers are linked to the customer's asset (don't worry about what an asset really is, it doesn't really matter) and are created at the same time the asset is created and from the same "create asset" page, so I used an inline formset to link the answers to the asset.  I thought this would be easy, but I ran into a number of problems and eventually just hand-wrote the forms.  But I'll go through the process here really for my own sake of remembering later what the limitations of inline formsets are.

models.py:
class Question(models.Model):
    question = models.CharField(max_length=250, editable=False)

    def __unicode__(self):
        return unicode(self.question)

class Answer(models.Model):
    asset = models.ForeignKey(Asset)
    question = models.ForeignKey(Question)
    answer = models.BooleanField(default=False)

    def __unicode__(self):
        return unicode('%s: %s' % (self.question, self.answer))


My tables actually had more to them, but for the sake of this example I've simplified them.

forms.py
class AnswerForm(ModelForm):
    to_save = False
    question = forms.ModelChoiceField(queryset=Question.objects.all(),
                                      widget=forms.HiddenInput())

    class Meta:
        model = Answer
        exclude = ('asset')

    def set_question(self, question):
        self.fields['answer'].label = question

views.py (there was a separate method for saving the formset, but I've unfortunately lost that code)
def get_answer_formset(asset, data):
    questions = StageQuestion.objects.all()
    extra = 0
    if not asset:
        # This is a brand new set, so we should pre-populate the questions.
        extra = len(questions)

    AnswerFormSet = inlineformset_factory(
            Asset,
            Answer,
            form=AnswerForm,
            can_delete=False,
            extra=extra,
            )

    formset = AnswerFormSet(
            data,
            prefix='que',
            instance=asset,
            )

    for form, question in zip(formset.forms, questions):
        form.set_question(question.question)
        form.initial = {'question': question.id}

    return formset

This code probably doesn't work too well anymore, since I changed it so many times, but here's a list of issues I ran into. The way a formset works is you tell it how many extra empty entries you want to display. When you have a brand new asset, extra would simply equal the number of Answers total, since I don't allow the user to actually add or remove Answers. So at first I set `extra` to equal the number of questions I wanted the user to answer, and I pre-populated each answer form with a unique question. This worked for creation, but when I wanted to edit an existing asset, it showed a list of all the questions and then a list of all the questions again, because it turned out `extra` is the number of (duh) *extra* empty questions to show. So then I set `extra` to zero, thinking that I don't want any *extra* empty questions, I just want the set of original questions, but you can't pre-define objects before putting them in the formset, so my icky solution was to set `extra` to zero only when an asset already exists and to set it to the number of questions otherwise. This worked, but it's certainly not clean.

By the way, look at that last part with the zip. This is pretty cool. I needed to loop through each form and seed it with the question. The `form.set_question` part sets the answer label so it correctly displays the question in the html. Django doesn't seem to have a label form, just a label attribute of each type of form, so I had to set the answer label to the question text and then hide the question field, otherwise the question would show up as a dropdown list of questions to select from. The `form.initial` part is my way of forcing initial data into the inline formset, since the init method doesn't allow it. This bothers me, actually, because regular formsets allow initial data, so why not inline formsets (especially since inline formsets subclass regular formsets)?

Back to the final issue. Formsets are "smart" enough to know not to add empty objects to the database, and an unchecked checkbox looks exactly like an empty object. This meant that only the True answers were saved in the database. This wouldn't seem so bad, after all it's saving DB space, but then if the user wants to edit their asset and maybe change some of their answers, you need to add back in the questions that they previously answered False on, and that's just ugly. Alternatively, you could change the saving of the formset to save the empty forms too, but I tried this and started running into issues with the forms missing key data, and finally I decided that I was hacking formsets so much that they were no longer useful. So I changed it to a list of forms. Trying to get formsets working took me a couple of days. The list of forms was done quickly and cleanly. When you look at it, it's clear what it's doing, and no more ugly hacks!

In conclusion, while it's possible none of this was clearly written enough to really understand, just know that formsets are great for what they were made for: allowing the user to add multiple objects of the same type; they are not good for a static list of objects that need to be created.

Monday, June 20, 2011

.screenrc

Here is my .screenrc file:

escape ^Jj
startup_message off
defscrollback 10000
vbell off
autodetach on
hardstatus alwayslastline
#hardstatus string "%{=b}%{G} %{b}%w%=%{kG}%C%A"
hardstatus string '%{= kG}[%{G}%{y}%H%{g}][%= %{= kw}%?%-Lw%?%{r}(%{W}%n*%f%t%?(
%u)%?%{r})%{w}%?%+Lw%?%?%= %{g}][%{B}%m/%d %{W}%c%{g}]'
shell -$SHELL

.vimrc

Here is my .vimrc file:


" Whitespace
set expandtab           " tabs -> spaces
set tabstop=4           " tab = 4 spaces
set shiftwidth=4        " used with << and >> to shift text over

set showmode            " when in INPUT mode, show INPUT text at all times
set ruler               " show current position at all times
set showcmd             " show incomplete commands at bottom of screen

set title               " set title to name of open file
set showmatch           " show matching brace/parenthese/bracket
" make the matching parenthesis not stand out so much
hi MatchParen cterm=underline ctermbg=black

" after leaving VIM, remove that annoying 'Thanks for flying Vim' title
let &titleold="~"

set visualbell          " flash screen instead of beep when error happens

" Search
set incsearch           " start searching as you type the search
set hls                 " highlight search matches
set ignorecase          " ignore case while searching
set smartcase           " if search string has uppercase, don't ignore case
set infercase           " if using auto-completion, this makes the word found have an appropriate case
" use spacebar to clear search highlighting and any message already displayed
nnoremap :silent nohecho

syntax on               " turn on syntax coloring
"syntax match Test "p"
"highlight Test ctermbg=red guibg=red
"syntax match Tab "\s"
"highlight Tab ctermbg=red guibg=red

au BufReadPost,FileReadPost * syntax match OverLength "\%>79v.\+"
hi OverLength ctermbg=blue ctermfg=white guibg=blue

au BufReadPost,FileReadPost * syntax match TrailingSpaces "\s\+$"
hi TrailingSpaces ctermbg=red guibg=red

" highlight text past 79 characters
match OverLength /\%>79v.\+/

" highlight trailing spaces at end of line
"2match TrailingSpaces /\s\+$/

"highlight TrailingSpaces ctermbg=red guibg=red
highlight OverLength ctermbg=blue ctermfg=white guibg=blue

" Function to remove superfluous white space from the end of a line
function! RemoveWhiteSpace()
    :%s/\s*$//g
    :'^
    "`.
endfunction

Cygwin Tweak

to get a cygwin terminal that actually uses bash and allows you to paste stuff into the window, edit /cygdrive/c/cygwin/Cygwin.bat to have this:


@echo off

C:
chdir C:\cygwin\bin

start rxvt -sr -sl 10000 -fg grey -bg black -fn courier -tn cygwin -e /bin/bash --login -i