refactor: moved to hugo

content/blog/2013/07/01/django-stampu-static-sites-with-django/index.md

@@ -0,0 +1,41 @@
++++
+title = "django-stampu: Static sites with django"
+date = 2013-07-01
++++
+
+![](django-stampu.png)
+
+> After finishing zombiepress a friend of mine wanted to use it but asked for
+some kind of conversion to static since he uploads its current blog to github.
+That's how this project was born.
+
+Django as a static site? Of course, it's possible, only if your project it's
+using a simple backend infrastructure, it doesn't let users modify the
+database, and also doesn't have contact forms or that kind of stuff. The
+conversion of a blog into static content was really easy, in about 30 minutes
+I had an alpha working -which is the 0.1.2 version uploaded in github/pypi-.
+
+django-stampu is basically a crawler. Some people led me to solutions much
+more complicated. (Ex. Running a command and let the computer work.) And I
+didn't like that. I wanted a conversion of the site as I see it when running
+the `runserver` command, for that purpose you have to follow all the internal
+links of the site and save the items as *.html if the content-type matches. It
+also saves the static files in the site, but I need to look further into that,
+because it can't save files needed by css files yet, it only checks for
+src/href attributes on all tags in html files.
+
+Want to try it?
+
+  1. Install django-stampu: `pip install django-stampu`
+  2. Add it to your `INSTALLED_APPS`
+  3. Execute the command: `python manage.py stamp`
+  4. Check the `_static` folder.
+
+Easy, right? That's what I want to hear!
+
+Keep in mind that this is a proof of concept, I need to improve lots of things
+and add some configuration variables for people that want to customize their
+sites.
+
+If you want to contribute in any way, you can [check the source code at
+github](https://github.com/fmartingr/django-stampu).

content/blog/2013/07/04/extracting-data-from-obfuscated-java-code/index.md

@@ -0,0 +1,207 @@
++++
+title = "Extracting data from obfuscated java code"
+date = 2013-07-04
++++
+
+![](header.png)
+
+For those who don't know, I started a site a while ago minecraft related (yes,
+[the one I dropped](/blog/2013/1/12/weekly-project-status-dropping-projects-
+hard/)). If you don't know what minecraft is (really?!), you can check [the
+official site](http://minecraft.net/), since this game can be a little
+difficult to explain.
+
+The project (which is online at
+[minecraftcodex.com](http://www.minecraftcodex.com)) is just a database of
+items, blocks, entities, etc. related to the game, but as in any other site of
+this kind, entering all this information can lead to an absolute _boredom_. So
+I thought... what if I can extract some of the data from the game
+_classfiles_? That would be awesome! *Spoiler alert* I did it.
+
+> Think this as my personal approach to all steps below: it doesn't mean that they're the best solutions.
+
+## Unpackaging the jarfile and decompiling the classes
+
+First of all, you have a _minecraft.jar_ file that it's just a packaged set of
+java compiled files, you can just `tar -xf` or `unzip` it into a folder:
+
+``` text
+unzip -qq minecraft.jar -d ./jarfile
+```
+
+With this we now have a folder called _jarfile__ _filled with all the jar
+contents. We now need to use a tool to decompile all the compiled files into
+.java files, because the data we're looking for it's hard-coded into the
+source. For this purpose we're going to use [JAD](http://varaneckas.com/jad/),
+a java decompiler. With a single line of _bash_ we can look for all the .class
+files and decompile them into .java source code:
+
+``` text
+ls ./jarfile/*.class | xargs -n1 jad -sjava -dclasses &> /dev/null
+```
+
+All the class files have been converted and for ease of use, we've moved them
+into a separate directory. But there's a lot of files! And also, when we open
+one...
+
+``` java
+public class aea extends aeb
+{
+    public aea()
+    {
+    }
+
+    protected void a(long l, int i, int j, byte abyte0[], double d,
+            double d1, double d2)
+    {
+        a(l, i, j, abyte0, d, d1, d2, 1.0F + b.nextFloat() * 6F, 0.0F, 0.0F, -1, -1, 0.5D);
+    }
+    // ...
+}
+```
+
+Look at that beautiful obfuscated piece of code! This is getting more
+interesting at every step: almost 1.600 java files with obfuscated source
+code.
+
+## Searching for the data
+
+I took the following approach: Since I know what I'm looking for (blocks,
+items, etc) and I also know that the information is hard-coded into the
+source, there must be some kind of string I can use to search all the files
+and get only the ones that contains the pieces of information I look for. For
+this test, I used the string "diamond":
+
+``` text
+$ grep diamond ./classes/*
+./classes/bfp.java:        "cloth", "chain", "iron", "diamond", "gold"
+./classes/bge.java:        "cloth", "chain", "iron", "diamond", "gold"
+./classes/kd.java:        w = (new kc(17, "diamonds", -1, 5, xn.p, k)).c();
+./classes/rf.java:        null, "mob/horse/armor_metal.png", "mob/horse/armor_gold.png", "mob/horse/armor_diamond.png"
+./classes/xn.java:        p = (new xn(8)).b("diamond").a(wh.l);
+./classes/xn.java:        cg = (new xn(163)).b("horsearmordiamond").d(1).a(wh.f);
+```
+
+As you can see, with a simple word we've filtered down to five files (from
+1.521 in this test). Is proof that we can get some information from the source
+code and we now to filter even more, looking around some files I selected
+another keyword: _flintAndSteel_, works great here, but in a real example you
+will need to use more than one keyword to look for data.
+
+``` text
+$ grep flintAndSteel ./classes/*
+./classes/xn.java:    public static xn k = (new xh(3)).b("flintAndSteel");
+```
+
+Only one file now, we're going to assume that all the items are listed there
+and proceed to extract the information.
+
+## Parsing the items
+
+This was the more complicated thing to do. I started doing some regular
+expressions to matchs the values I wanted to extract, but soon that became
+inneficient due to:
+
+- The obfuscated code varies with every released version/snapshot -or it should.
+- The use of OOP difficulted method searching with RegEx matching, since the names could change from version to version, making the tool unusable on updates.
+- The need to modify the RegEx if something in the code changes, or if we want to extract some other value.
+
+After some tests, I decided to _convert_ the java code into python. For that,
+I used simple find and match to get the lines that had the definitions I
+wanted, something line this:
+
+``` java
+// As a first simple filter, we only use a code line if a double quote is found on it.
+// Then, regex: /new (?P<code>[a-z]{2}\((?P<id>[1-9]{1,3}).*\"(?P<name>\w+)\"\))/
+// ...
+T = (new xm(38, xo.e)).b("hoeGold");
+U = (new yi(39, aqh.aD.cE, aqh.aE.cE)).b("seeds");
+V = (new xn(40)).b("wheat").a(wh.l);
+X = (vr)(new vr(42, vt.a, 0, 0)).b("helmetCloth");
+Y = (vr)(new vr(43, vt.a, 0, 1)).b("chestplateCloth");
+// ...
+```
+
+Since that java code is not python evaluable, just convert it:
+
+- Remove unmatched parenthesis and double definitions
+- Remove semicolons
+- Remove variable definitios
+- Converted arguments to string. This can be improved a lot, leaving decimals, converting floats to python notation, detecting words for string conversion, etc. Since for now I am not using any of the extra parameters this works for me.
+- Be careful with reserved python names! (`and`, `all`, `abs`, ...)
+
+``` python
+// Java: U = (new yi(39, aqh.aD.cE, aqh.aE.cE)).b("seeds");
+yi("39", "aqh.ad.cE", "aqh.aE.cE").b("seeds")
+// Java: bm = (new xi(109, 2, 0.3F, true)).a(mv.s.H, 30, 0, 0.3F).b("chickenRaw");
+xi("109", "2", "0.3F", "true").a("mv.s.H", "30", "0", "0.3F").b("chickenRaw")
+```
+
+Now I defined an object to match with the java code definitions when
+evaluating:
+
+``` python
+class GameItem(object):
+    def __init__(self, game_id, *args):
+        self.id = int(game_id)
+
+    def __str__(self, *args):
+        return "<Item(%d: '%s')>" % (
+            self.id,
+            self.name
+        )
+
+    def method(self, *args):
+        if len(args) == 1 and isinstance(args[0], str):
+            "Sets the name"
+            self.name = args[0]
+        return self
+
+    def __getattr__(self, *args):
+        return self.method
+```
+
+As you can see, this class have a global "catch-all" method, since we don't
+know the obsfuscated java names, that function will handle every call. In that
+concrete class, we now that an object method with only one string parameter is
+the one that define the item's name, and we do so in our model.
+
+Now, we will evaluate a line of code that will raise and exception saying that
+the class name _&lt;insert obfuscated class name here&gt;_ is not defined.
+With that, we will declare that name as an instance of the GameItem class, so
+re-evaluating the code again will return a GameItem object:
+
+``` python
+try:
+    # Tries to evaluate the piece of code that we converted
+    obj = eval(item['code'])
+except NameError as error:
+    # Class name do not exist! We need to define it.
+    # Extract class name from the error message
+    # Defined somewhere else: class_error_regex = re.compile('name \'(?P<name>\w+)\' is not defined')
+    class_name = class_error_regex.search(error.__str__()).group('name')
+    # Define class name as instance of GameItem
+    setattr(sys.modules[__name__], class_name, type(class_name, (GameItem,), {}))
+    # Evaluate again to get the object
+    obj = eval(item['code'])
+```
+
+And with this, getting data from source code was possible and really helpful.
+
+A lot of things could be improved from this to get even more information from
+the classes, since after spending lot's of time looking for certain patterns
+on the code I can say what some/most of the parameters mean, and that means
+more automation on new releases!
+
+## Real use case
+
+Apart from getting the base data for the site (all the data shown on minecraft
+codex is directly mined from the source code), I made up a tool that shows
+changes from the last comparision -if any. This way I can easily discover what
+the awesome mojang team added to the game every snapshot they release:
+
+![](diff.png)
+
+This is the main tool I use for minecraft codex, is currently bound to the
+site itself but I'm refactoring it to made it standalone and publish it on
+github.

content/blog/2013/07/09/how-install-and-configure-yubikey-pam-module-archlinux/index.md

@@ -0,0 +1,104 @@
++++
+title = "How to install and configure the yubikey-pam module on archlinux"
+date = 2013-07-09
++++
+
+![](yubico.jpg)
+
+Not so long ago I've been gifted with a Yubikey. It's a two-auth hardware
+solution with multiple auth methods (OTP, Challenge-response, OATH-HOTP and
+static password). It easily scaled to one of my favourite and most useful
+gadgets.
+
+I've been a google authenticator user for a while, but the yubikey is just
+much easier to work with: when configured, you only need to tap a button on
+the usb stick to generate your key. You can use this in many ways, but in this
+post I'm focusing on ssh authentication and how to install the yubico-pam
+module into an Arch installation.
+
+## Installing the required packages
+
+Thanks to the awesome arch community we already have the needed packages on
+the AUR, these are: [yubico-pam-
+git](https://aur.archlinux.org/packages/yubico-pam-git/), [yubico-c-client-
+git](https://aur.archlinux.org/packages/yubico-c-client-git/), [yubikey-
+personalization-git](https://aur.archlinux.org/packages/yubikey-
+personalization-git/) and
+[libyubikey](https://aur.archlinux.org/packages/libyubikey/). Keep in mind
+that you _may_ need to install more packages depending on your system
+installation.
+
+You can install that with your favourite AUR helper or using `makepkg`:
+
+``` text
+$ curl -O https://aur.archlinux.org/packages/li/libyubikey/libyubikey.tar.gz
+$ tar xvzf libyubikey.tar.gz
+$ cd libyubikey
+$ makepkg PKGBUILD
+# ...
+$ sudo pacman -U libyubikey-1.10-2-x86_64.pkg.tar.xz
+```
+
+Repeat that step for all the packages, in order: _libyubikey_, _yubico-c-
+client_, _yubikey-personalization_ and _yubico-pam_. If you have trouble
+installing from the AUR [refer to the appropiate wiki
+page](https://wiki.archlinux.org/index.php/AUR#Installing_packages).
+
+## Configure the PAM module
+
+Edit `/etc/pam.d/sshd` and add on top on the rest of the auth modules:
+
+``` text
+auth sufficient pam_yubico.so id=XXXX key=XXXX
+```
+
+You can obtain an ID/key conbination by registering your yubikey [at this
+page](https://upgrade.yubico.com/getapikey/).
+
+## Authorization methods
+
+### Individual authorization mapping
+
+If your server have multiple users this is the easiest method to let them
+configure their yubikeys. You just need to create the file
+`$HOME/.yubico/authorized_yubikeys` with the following contents:
+
+``` text
+<username>:<Yubikey token ID 1>[:<Yubikey token ID 2][:...]
+```
+
+The yubikey token identifier can be obtained by removing the last 32
+characters of any OTP value, and you can add more than one ID to the file.
+
+Restart your ssh server to apply the changes.
+
+### Central authorization mapping
+
+Create a file on `/etc/yubikey_mappings` that will contain all your users and
+their respective yubikey token identifiers, like this:
+
+``` text
+<first username>:<Yubikey token ID 1>[:<Yubikey token ID 2][:...]
+<second username>:<Yubikey token ID 3>[:<Yubikey token ID 4][:...]
+```
+
+For this to work, you need to specify this file to the pam module `authfile`
+parameter:
+
+``` text
+auth sufficient pam_yubico.so id=XXXX key=XXXX authfile=/etc/yubikey_mappings
+```
+
+After that estart your ssh server to apply the changes.
+
+## Logging in
+
+The next time you're asked for a password on you ssh login you can use a
+yubikey OTP instead of your current password -if you have any-.
+
+This method works pretty well with authorized ssh keys as well, since you will
+log-in automatically from a computer with a configured ssh key but an OTP -or
+password- will be required for logging in from anywhere else.
+
+**Yubico-pam module |** [Github](https://github.com/Yubico/yubico-pam)
+**Yubico |** [Home page](http://www.yubico.com/)