25 Years, 5 Months, and 14 days

02 Oct 2014

Today I am 25 years, 5 months, and 14 days old, the exact same age my brother was when he passed away. This day has been on my mind everyday for the past year. I've spent long car rides, sleepness nights, and countless laps swimming thinking about today, asking my self what should I do, what should I say. After wrestling with my thoughts, I thought it may be best to write out my thoughts and reflect.

My brother, Harel, had an incredible abillity to touch everyone that he came in contact with. I remember when I was younger it felt like he knew everyone. There was not a place I would go with my brother where we wouldn't randomly bump into someone he knew. Even more so, the respect and excitement that was shown to my brother by everyone around him was incredible. When my brother would walk into a room, it was as if all the attention just shifted to him and regardless of what the mood was before, it immediately got better.

Despite many of the hardships my brother had growing up, from not having much growing up to losing friends as a teenager, he had this incredible ability to flip a situation and look at the bright side. He was always positive and knew how to make the best of any situation. His smile was contagious, his stories were great, and just hearing his laugh could brighten up your day.

I think, to me, what may have cemented who my brother was, was shown on the day of his funeral. The sheer amount of people from such a wide array of groups who came to pay my brother respect was incredible. In such a short lifetime, my brother was able to touch the lives of so many, something I truly admire him for.

A quote that I came across a few years ago has stuck with me.

There are moments in each of our lives when something so dramatic happens that one can barely remember what life was like before. These moments reshape the prism through which we can see everything that follows. These moments define the chapters in our lives, and how we react to them defines who we are.

For me, this moment is when my brother passed away. I feel as if a hard line was drawn through my life on the day of his passing. Coping with my brother's death has been the single most difficult thing I have had to do. Bad days become worse and no matter how much time passed, it didn't feel like the pain was going away. Some days, I hate the world for what it has taken from me. It seems like there are days when I can't remember ever truly being happy.

But, I refuse to let these bad days get the best of me. I refuse to forget the good times I had with my brother and everything he taught me. My whole life, my brother has been my role model, inspiration, and motivation. Until this day, I still want to be like Harel when I grow up. I realize now, if I truly want to be like my brother when I grow up, then I cannot forget what my brother stood for.

So my brother, wherever you may be, I hope you can read this, see this, or hear this and I hope that I've made you proud. I hope that one day I can positively influence the lives of as many people you did throughout your lifetime. Hopefully. one day, HK, I'll be like my big brother.

Harel and Barak


Play Framework: Returning JSON Responses

06 Apr 2014

I've recently started a project at working using the Play Framework and while its a great framework, I was having a lot of trouble with some of the simplest tasks. I wouldn't blame Play for my problems, returning to Java after a long hiatus, being spoiled by dynamicly typed languages, and lack of documentation really made such tasks like returning a JSON response difficult.

I figured that I may not be the only in this position, judging by the questions in the IRC channel and lack of responses, I figured it may be a good idea to jot down some of my notes, not only for myself, my coworkers, but for all my fellow Play framework noobs. Apologies for the introduction, I'll try to keep the rest of this posts and the futures, short, succinct, and to the point.

tl;dr

Use ObjectNode class found in the com.fasterxml.jackson.databind.node.ObjectNode package.

import com.fasterxml.jackson.databind.node.ObjectNode;

    ObjectNode response = Json.newObject();
    response.put("status", 200);
    response.put("message", "Request was successful");
    response.put("data", Json.toJson(users));
    return ok(response);

Using ObjectNode

To easily return mixed response types, I suggest using the ObjectNode class found in the com.fasterxml.jackson.databind.node.ObjectNode package. Using the class is simple, first instantiate an object:

ObjectNode response = Json.newObject();

From there, you can treat the object similar to a map but using put() function.

response.put("status", 200);
    response.put("message", "Request was successful");

Often times we want to return an object or array of objects. Using the Json.toJson() function found in play.libs.Json we can easily convert objects or lists of objects to be included in our JSON response.

response.put("data", Json.toJson(users));

Finally, you'll probably want to use the static return response methods to easily return our JSON response.

return ok(response);


Play Framework: Parsing Arrays in JSON Requests

06 Apr 2014

tl;dr

I suggest using ObjectMapper found in the com.fasterxml.jackson.databind.ObjectMapper

ObjectMapper mapper = new ObjectMapper();
    try {
        JsonNode root = request().body().asJson();
        JsonNode jsonPhoneNumbers = root.get("phoneNumbers");
        for(JsonNode phoneNumberString : jsonPhoneNumbers) {
            String phoneNumber = phoneNumberString.asText();
        }
    catch (NullPointerException npe) {
        // No valid phone numbers were provided
    }

Using ObjectMapper

For parsing arrays in JSON requests, I suggest using the ObjectMapper. Assuming you have a request that looks like:

{
        phoneNumbers: [
            '555-555-5555',
            '555-555-5556',
            '555-555-5557'
        ]
    }

We can easily access the phone number values by using ObjectMapper class. The first thing that you'd want to do is instantiate an instance of this class, which is pretty straight forward.

ObjectMapper mapper = new ObjectMapper();

From there, we want a JSON version of our request, the Play framework allows you to easily do so by chaining the asJson() method to the request().body() functions.

JsonNode root = request().body().asJson();

Using the get() function on our root object and providing the correct key, we can easily get our phoneNumbers as a JsonNode, which is also iterable. By iterating through each of the JsonNode objects, we can just return the value for the corresponding phone number as a String by using the asText() method.

JsonNode jsonPhoneNumbers = root.get("phoneNumbers");
    for(JsonNode phoneNumberString : jsonPhoneNumbers) {
        String phoneNumber = phoneNumberString.asText();
    }

Since there is a possibility that the request could be empty (null), I suggest wrapping the request processing in a try-catch block. If a request is null or the specified key was not included in the request (in this example it would be phoneNumbers) then we can use the catch portion of the block to easily return a bad request.

Put this all together looks like:

import com.fasterxml.jackson.databind.JsonNode;
    import com.fasterxml.jackson.databind.ObjectMapper;
    import com.fasterxml.jackson.databind.node.ObjectNode;
    import play.libs.Json;

    ObjectMapper mapper = new ObjectMapper();
    ObjectNode response = Json.newObject();
    try {
        JsonNode root = request().body().asJson();
        JsonNode jsonPhoneNumbers = root.get("phoneNumbers");
        for(JsonNode phoneNumberString : jsonPhoneNumbers) {
            String phoneNumber = phoneNumberString.asText();
        }
    catch (NullPointerException npe) {
        // No valid phone numbers were provided
        response.put("status", 400);
        response.put("message", "No phone numbers were provided");
        return badRequest(response);
    }


Counter Resets and Database Magic

08 Dec 2013

Recently, the need for a standard numbering convention came up for one of our projects at work. The client requested to be able to access protocols by a unique ID. Originally I thought it'd be a no brainer, I'd explain to them that the each protocol is already uniquely identifiable within the database, and they can just refer to protocols by their unique ID in the database. Unfortunately, they didn't seem too fond of the unique ID numbering scheme and proposed that the application follow a certain convention, which they created.

The convention was simple enough to understand. It followed the format of:

YYYY-XXX

Where YYYY was the Academic Year and XXX was a 3 digit number signifying that it was the nth protocol scheduled for review of that academic year. For example, a protocol with the number

1314-016

Would be the 16th protocol scheduled within the 2013-2014 academic year.

While the standard is easy enough, it brings up some concerns.

  1. Protocol numbers are generated once a protocol has been scheduled for committee review.
  2. Protocol numbers must refer to a specific protocol.
  3. The protocol number must reset after every academic year.
  4. How can I enforce this standard within the database?

Regarding the first concern, I couldn't simply append a column to the protocol table since I would have a NULL field until the protocol was scheduled (if it was ever scheduled). While adding a column would allow me to specify which number belonged to which protocol, it invalidate the 3NF standard I'm trying to achieve. Also, how would I have the database automatically increment or reset the value of the counter?

I realized quickly that I needed another table (or two). Immediately, I realized that my protocol_number table would need the following columns:

  • ID - mainly for reference
  • Protocol ID - specifies which number this protocol refers to
  • Academic Year - the year part of the number convention
  • Protocol Number - the counter part of the number convention

While designing the table, I also realized that I didn't want to have to worry about the application needing to worry about which protocol number to insert within the column and that lead me to the idea of a trigger. I know that I can easily count the number of rows within a table using the SQL command count(*), but how could I use it for handling the protocol number field? I realized that my answer would rely in a trigger that would fire before each insert on the table.

Before jumping into creating the trigger, I had to give some more thought to the the academic year field. While I could easily represent academic years with an INT field, it didn't seem to be sufficient, something just felt dirty. I made the decision that I'd create a new table, academic_year, which would simply store academic years. This way, I could have the protocol_number table refer to the academic_year table as a foreign key. Specifying the field as a foreign key also provided me the added benefit of being able to lookup which protocols were scheduled for what year (a feature that hasn't been asked for, but can now be easily implemented).

Given this analysis, we've composed two tables with the following structure:

CREATE TABLE academic_year(
    id int UNSIGNED NOT NULL AUTO_INCREMENT,
    academic_year int UNSIGNED NOT NULL,
    PRIMARY KEY(id)
);

CREATE TABLE protocol_number(
    id int UNSIGNED NOT NULL AUTO_INCREMENT,
    academic_year_id int UNSIGNED NOT NULL,
    protocol_id int UNSIGNED NOT NULL,
    protocol_number int UNSIGNED NOT NULL,
    FOREIGN KEY (protocol_id) REFERENCES protocols(id),
    KEY (id)
);

Almost complete. We still have one tiny little problem. We want to ensure that our protocol numbers are unique and also that they're unique to a single protocol. Using primary keys we can specify that our protocol_number and academic_year_id columns are unique. In addition, we can easily add a unique key on our protocol_id field allowing us to ensure that this field is also unique. Doing so creates the following structure:

CREATE TABLE protocol_number(
    id int UNSIGNED NOT NULL AUTO_INCREMENT,
    academic_year_id int UNSIGNED NOT NULL,
    protocol_id int UNSIGNED NOT NULL,
    protocol_number int UNSIGNED NOT NULL,
    FOREIGN KEY (protocol_id) REFERENCES protocols(id),
    UNIQUE KEY unique_protocol_id (protocol_id),
    FOREIGN KEY (academic_year_id) REFERENCES academic_year(id),
    PRIMARY KEY (protocol_number, academic_year_id),
    KEY (id)
);

Finally, we can create our trigger to handle incrementing the counter. Our trigger will be based around the query

SELECT COUNT(*) as num_protocols 
FROM eirb_number 
WHERE academic_year_id=[ACADEMIC_YEAR_ID]

The query will return the number of rows that exist with the given ACADEMIC_YEAR_ID. All we would have to do now is take the results of this query, increment it by 1, and then set it as the value for the protocol_number column for the new row being inserted into the database. Given our analysis it sounds like we have all we need to create our a trigger, that is, when it occurs, BEFORE INSERT and what we want to add/change protocol_number with our count(*) value.

Behold, our trigger:

delimiter //
CREATE TRIGGER insert_protocol_num_trigger 
BEFORE INSERT on eirb_number
for each row
BEGIN
    SET new.protocol_number = (
        SELECT COUNT(*) as num_protocols 
        FROM eirb_number 
        WHERE academic_year_id=new.academic_year_id
    ) + 1;
END; //
DELIMITER ;

Now all our queries/application have to worry about is what protocol they want to create the number for and which academic year that protocol belongs to. The database will handle the rest. Our insert query can now look something like:

INSERT INTO protocol_number(protocol_id, academic_year_id) VALUES(4, 1);

Now we've handled the case of reset counters through the use of database magic (AKA triggers).


More TDD and Recursion Practice

24 Nov 2013

It seems that Javascript challenges are more frequent at work. Thursday afternoon when we got back from lunch, my coworker was testing himself in the You Can't Javascript Under Pressure challenges. As great coworkers, we (by that I mean, me) decided to help out and put more pressure on our fellow coworker, without even being asked to! One of the challenges that came up during the test was a function ArraySum() which accepts a list (array) of values and you must total all the integers within the array. The array could contain strings, numbers, booleans, lists of strings, numbers, booleans, etc. Regardless of the input, you must total all the integers within the array. After seeing that challenge (and helping with the correct answer), I thought it might be good practice for another blog post.

Practice makes Perfect

Before jumping into more testing and programming, I want to stop and think about my methodology. While re-reading Chapter 3 of Test-Driven Web Development with Python on Friday with a coworker, I realized that almost every line of code that was written in the chapter to satisfy the test, had a test case as a reason to write that code. After seeing this and thinking about my previous blog post, I realized I didn't follow the TDD methodology as closely as I couldshould have. According to the author, Harry Percival, TDD is a discipline, it even says so in chapter 4.

TDD is a discipline, and that means it's not something that comes naturally; because many of the payoffs aren't immediate but only come in the longer term, you have to force yourself to do it in the moment.

To cut myself some slack, TDD is something you have to practice. Yes, I'm talking about practice. I'm sure there are a bunch of young grasshoper learning quotes I could state right now, but lets get on to testing.

Stricter Test Cases

Last time we started by importing our function and writing a basic test case to test the return type. Let's do that again.

import unittest
from arraysum import ArraySum

class ArraySumTests(unittest.TestCase):
    def setUp(self):
        pass

    def test_for_int(self):
        result = ArraySum([1, 2, 3, 4, 5,])
        self.assertTrue(type(result) is int, "Result is not integer")

if __name__ == '__main__':
    unittest.main()

Looks good to me, lets run it!

Traceback (most recent call last):
File "tests.py", line 2, in <module>
  from arraysum import ArraySum
  ImportError: No module named arraysum

Our first error! Lets correct it, but remember, minimal amount of code! In fact, we won't right any code at all.

caster:arraysum/ $ touch arraysum.py

Let's re-run our test.

Traceback (most recent call last):
File "tests.py", line 2, in <module>
  from arraysum import ArraySum
  ImportError: cannot import name ArraySum

Yay new error! This time it can't find our function, ArraySum, lets (minimally) create that in our new arraysum.py file.

def ArraySum(int_list):
    pass

Okay, let's see what we get now.

F
======================================================================
FAIL: test_for_int (__main__.ArraySumTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests.py", line 10, in test_for_int
  self.assertTrue(type(result) is int, "Result is not integer")
  AssertionError: Result is not integer

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (failures=1)

MoreMOAR errors!! This time our test is failing because we're not returning an integer. Let's fix that by modifying our return statement.

def ArraySum(int_list):
    return 0

Alright, lets re-run our test.

.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Woohoo! Our first test passed. According to TDD, we can now write some more tests. Let's make this one a little more interesting.

import unittest
from arraysum import ArraySum

class ArraySumTests(unittest.TestCase):
    def setUp(self):
        pass

    def test_for_int(self):
        result = ArraySum([1, 2, 3, 4, 5,])
        self.assertTrue(type(result) is int, "Result is not integer")

    def test_flat_list(self):
        result = ArraySum([1, 2, 3, 4, 5])
        self.assertTrue(result == 15, "Incorrect sum")

if __name__ == '__main__':
    unittest.main()

We've now added a new test, test_flat_list to test a simple list of integers and ensure the sum is correct. Back to running tests...

F.
======================================================================
FAIL: test_flat_list (__main__.ArraySumTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 14, in test_flat_list
  self.assertTrue(result == 15, "Incorrect sum")
  AssertionError: Incorrect sum

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (failures=1)

The obvious problem here is that our function is returning 0 and not doing any sort of summation. According to TDD, we want to write the minimal amount of code, Python, being the beautiful language that it is, provides us with a simple sum function which operates on iterables. Allowing us to very easily pass our test_with_flat_list test.

def ArraySum(int_list):
    return sum(int_list)

Running our test again will show that we were able to successfully pass it.

..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

We can continue writing more test cases, so lets make it a little more interesting. We'll introduce some non-integer values into the list, which will create errors for our sum function in a test case called test_complex_list.

import unittest
from arraysum import ArraySum

class ArraySumTests(unittest.TestCase):
    def setUp(self):
        pass

    def test_for_int(self):
        result = ArraySum([1, 2, 3, 4, 5,])
        self.assertTrue(type(result) is int, "Result is not integer")

    def test_flat_list(self):
        result = ArraySum([1, 2, 3, 4, 5])
        self.assertTrue(result == 15, "Incorrect sum")
    
    def test_complex_list(self):
        result = ArraySum([1, 2, 3, "hello"])
        self.assertTrue(result == 6, "Incorrect sum")


if __name__ == '__main__':
    unittest.main()

Running our tests, returns the results:

E..
======================================================================
ERROR: test_complex_list (__main__.ArraySumTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests.py", line 17, in test_complex_list
  result = ArraySum([1, 2, 3, "hello"])
    File "/home/caster/Development/arraysum/arraysum.py", line 2, in ArraySum
        return sum(int_list)
        TypeError: unsupported operand type(s) for +: 'int' and 'str'

----------------------------------------------------------------------
Ran 3 tests in 0.000s

FAILED (errors=1)

Just as we expected! Let's make some corrections to our ArraySum function.

def ArraySum(int_list):
    sum = 0
    for x in int_list:
        if type(x) is int:
            sum += x
    return sum

After making our fix, we'll re-run our test and...

...
----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK

Looks good! Once again, passed test cases means we can write more tests, so lets give it a go one last time. We'll create a new test case, test_complex_nested_list, where we'll introduce a nested list into our previous complex list. A nested list within our function also brings in the sense of recursion! We'll want to take in account what we have, where we want to go, and if we've done something like this before (which we have). Currently we're iterating through all the values and if they're integers we'll add them to our sum variable. We're throwing away that is anything but an int. We know this is wrong though since the function must also include the values within nested lists, hence the embarrassingly recursive algorithm.

We know that we're going to have to iterate through the values in our list, so we understand that, but lets look at our cases:

  • Value is an int: Add to sum
  • Value is a list: Add the total of the list to sum
  • Value is neither: Ignore

Now that we have an idea of what we want to do, lets write our last test case.

import unittest
from arraysum import ArraySum

class ArraySumTests(unittest.TestCase):
    def setUp(self):
        pass

    def test_for_int(self):
        result = ArraySum([1, 2, 3, 4, 5,])
        self.assertTrue(type(result) is int, "Result is not integer")

    def test_flat_list(self):
        result = ArraySum([1, 2, 3, 4, 5])
        self.assertTrue(result == 15, "Incorrect sum")
    
    def test_complex_list(self):
        result = ArraySum([1, 2, 3, "hello"])
        self.assertTrue(result == 6, "Incorrect sum")

    def test_complex_nested_list(self):
        result = ArraySum([1, 2, 3, "hello", [4, 5]])
        self.assertTrue(result == 15, "Incorrect sum")

if __name__ == '__main__':
    unittest.main()

Before we make any changes to ArraySum, we have to run our test.

.F..
======================================================================
FAIL: test_complex_nested_list (__main__.ArraySumTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests.py", line 22, in test_complex_nested_list
      self.assertTrue(result == 15, "Incorrect sum")
      AssertionError: Incorrect sum

----------------------------------------------------------------------
Ran 4 tests in 0.000s

FAILED (failures=1)

Now that we have a failed test, we are given the okay by the testing goat to modify our code. Let's add our recursive call, so our function now looks like:

def ArraySum(int_list):
    sum = 0
    for x in int_list:
        if type(x) is int:
            sum += x
        elif type(x) is list:
            sum += ArraySum(x)
    return sum

Re-running our final test...

....
----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK

Ahhhhh :)


Code Challenge: Dictionary Flatten

17 Nov 2013

While browsing /r/javascript Thursday night, I came across a post regrading a mildly interesting interview question which had to do with flattening a Javascript object to a URI.

Two things immediately jumped into my mind:

  1. I have to show one of my coworkers.
  2. This would make a "midly interesting" Python challenge!

Friday morning, the first thing I did when I got into work was write the challenge on our white board and my solution in Python. Funny enough, once I came back from lunch, my coworker was writing his Javascript solution next to my Python one. It turns out that he saw the challenge Thursday night and even posted his solution to the thread.

I figured, I'd code up my solution and give it a shot. Both for recursive practice and writing test cases. So to start with some test driven development (TDD), I began by writing a simple test case.

First we'll define a class for our test cases:

import unittest
class FlattenTests(unittest.TestCase):

From there, we'll add a simple setUp method which creates a dictionary to use for testing:

import unittest

class FlattenTests(unittest.TestCase):
    def setUp(self):
        self.example = {
                "foo": {
                    "bar": {
                        "team": True,
                        "company": ["Bill", "Ted"]
                        }
                    }
                }

Now lets define a test case!

import unittest

class FlattenTests(unittest.TestCase):
    def setUp(self):
        self.example = {
                "foo": {
                    "bar": {
                        "team": True,
                        "company": ["Bill", "Ted"]
                        }
                    }
                }

    def test_for_dict(self):
        d = flatten(self.example, "")
        self.assertTrue(type(d) is dict)


if __name__ == '__main__':
    unittest.main()

The test case we've defined, test_for_dict, just determines if the return type of the flatten method (which we haven't written yet) is of type dictionary. Lets run our test case and see what we get!

ERROR: test_for_dict (__main__.FlattenTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 16, in test_for_dict
d = flatten(self.example, "")
NameError: global name 'flatten' is not defined
----------------------------------------------------------------------

Ahh what was that!? According to my understanding of TDD, this is a good thing! We can start writing code now. Lets define a new file flatten.py and create a flatten function that accepts a dictionary and string. Remember, according to TDD, we must right the most minimal amount of code that will pass our test.

def flatten(d_flat, key_string):
    pass

Well that was simple enough, lets run our test.

======================================================================
FAIL: test_for_dict (__main__.FlattenTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 17, in test_for_dict
self.assertTrue(type(d) is dict)
AssertionError: False is not true
----------------------------------------------------------------------

Hmm, seems like we didn't write enough code! Lets go back and update our flatten function.

def flatten(d_flat, key_string):
   return {}

Now that we've updated our code, lets run our test again.

caster:flatten/ $ python tests.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Looks good to me! Now that one of our tests has passed, we can now write another (more interesting) test! I'm not very creative at writing these incremental tests, so lets just get to it.

import unittest
from flatten import flatten

class FlattenTests(unittest.TestCase):
    def setUp(self):
        self.example = {
                "foo": {
                    "bar": {
                        "team": True,
                        "company": ["Bill", "Ted"]
                        }
                    }
                }

    def test_for_dict(self):
        d = flatten(self.example, "")
        self.assertTrue(type(d) is dict)

    def test_case_one(self):
        d = flatten(self.example, "")
        self.assertTrue(d["foo/bar/team"])
        self.assertTrue(d["foo/bar/company"] == ["Bill", "Ted"]) 

if __name__ == '__main__':
    import unittest

We've defined a new test case, test_case_one, (like I said, I'm not very creative), that'll run our flatten function and determine if we've received the correct output. Lets try running our test case now (and cross our fingers) to see if it fails.

caster:flatten/ $ python tests.py
E.
======================================================================
ERROR: test_case_one (__main__.FlattenTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 21, in test_case_one
self.assertTrue(d["foo/bar/team"])
KeyError: 'foo/bar/team'

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (errors=1)

Awesome! More good news! :) Before we jump into writing our function, lets think about what we want the function to do.

  • First we'll need to iterate through all the key/value pairs in the dictionary passed to the function.
  • We'll then need to determine if the value for the key we've iterated to is a dictionary or not.
    • If the value is a dictionary, we'll want to recursively call flatten and append our key to key_string so that we're keeping track of the key we want to eventually use.
  • The easy (or base) case, if our value is NOT a dictionary, simply add it to the dictionary with it's key append to key_string
  • Finally, once we've iterated through all the values, return the dictionary.

When we put it like that, it sounds easy! (Or so I hope...)

Lets take care of iterating through the dictionary, that sounds easy enough:

def flatten(d_flat, key_string):
    d = {}
    for k,v in d_flat.items():
        pass
    return d

Simple enough, we've declared an empty dictionary d and started iterating through the items in the dictionary. Lets take a look at our first case, if the value for a key is a dictionary:

def flatten(d_flat, key_string):
    d = {}
    for k,v in d_flat.items():
        if(type(v) is dict):
            pass
        else:
            pass
    return d

Okay we've added our if statement, now if it's true, we'll have to recursively call our function. Before we jump to make the recursive call, we need to ensure we're keeping track of our key_string value. To do so, we'll pass key_string with the current key value, k. We'll recursively call flatten with the inner dictionary stored in v.

def flatten(d_flat, key_string):
    d = {}
    for k,v in d_flat.items():
        if(type(v) is dict):
            d = flatten(v, key_string + k + "/")
        else:
            pass
    return d

Almost there! We now need to worry about our base case, so let's take care of that! Recall that we said if our value is not of type dictionary, we simple append it to our dictionary that we declared, d. Again, we have to keep in mind that we want to assign the correct key for our value, which would be our current key_string concatenated with the current key we're looking at, k. Once we create the right key, we'll just assign it to our value, v and return our dictionary, d.

def flatten(d_flat, key_string):
    d = {}
    for k,v in d_flat.items():
        if(type(v) is dict):
            d = flatten(v, key_string + k + "/")
        else:
            d[key_string + k] = v
    return d

So good so far, lets run our tests!

caster:flatten/ $ python tests.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

Perfect...or so we think. There's actually a problem in our algorithm, which we can expose by making our testing data little more robust. Lets update our setUp and our test_case_one functions so they look like this:

import unittest
from flatten import flatten

class FlattenTests(unittest.TestCase):
    def setUp(self):
        self.example = {
                "foo": {
                    "bar": {
                        "team": True,
                        "company": ["Bill", "Ted"]
                        }
                    },
                    "fizz": {
                        "blue": "red"
                    }
                }

    def test_for_dict(self):
        d = flatten(self.example, "")
        self.assertTrue(type(d) is dict)

    def test_case_one(self):
        d = flatten(self.example, "")
        self.assertTrue(d["foo/bar/team"])
        self.assertTrue(d["foo/bar/company"] == ["Bill", "Ted"]) 
        self.assertTrue(d["foo/bar/fiz/blue"] is "red")

if __name__ == '__main__':
    unittest.main()

We've added another key that points to another dictionary. Nothing unusual or out of the oridinary. We've also added a new assertion to ensure that the key foo/bar/fiz/blue contains the string red. Lets run our test case and see what happens.

caster:flatten/ $ python tests.py
E.
======================================================================
ERROR: test_case_one (__main__.FlattenTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 21, in test_case_one
self.assertTrue(d["foo/bar/team"])
KeyError: 'foo/bar/team'

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (errors=1)

Failed! What's going on here? If you take a look at our algorithm, we're not updating our dictionary when we return from a recursive call, we're actually overwriting it. Let's make a small change and ensure we're updating our dictionary and see what happens.

def flatten(d_flat, key_string):
    d = {}
    for k,v in d_flat.items():
        if(type(v) is dict):
            d.update(flatten(v, key_string + k + "/"))
        else:
            d[key_string + k] = v
    return d

Now lets run our tests.

caster:flatten/ $ python tests.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

All is well and now our function works as expected!


Why Barakyo?

12 Nov 2013

Over this past weekend I was asked, twice, "Why Barakyo?" While the explanation is simple enough, I sometimes get the response that my Barakyo nickname isn't professional. I've even been told I should consider choosing another domain name. While I understand that my nickname isn't professional and even though I share my blog with companies that I am interested in, I didn't want to strip my blog of me, hence the reason for the personal category. One of my goals for my blog was that it would still express me, without having to be techincal. One way of doing that is through the use of my domain name, to me it's fun, expressive, and short! It's also pretty easy to spell over the phone :)

Legality

One tangent before I do jump into my nickname. There is some slight confusion on the spelling of my actual name (Barak or Barack). Legally, my name is spelled Barack. According to my parents, somewhere in my childhood, I, myself, decided that the correct spelling for my name was Barak. I don't personally remember when I made the switch, but from what I can remember I've been spelling my name Barak my whole life. Eventually, I'll get around to changing my name legally so I have a uniform spelling.

History: AOL Instant Messanger

It seems like now is a better time than ever to actually introduce the history of my nickname, Barakyo. When my friends and I were in middle school and high school, AOL Instant Messanger (AIM) was the hip messenger to be on, EVERYONE was on it. Cool nicknames and aliases were rampant, but my friends and I had a different idea in mind. Young rebels at heart, we went against the grain, we decided to actually INCLUDE our names in our AIM screen names. Originally our screen names started out as, quack its robert and woof its barak. But this level of rebelling wasn't enough for us, we need moreMOAR.

Somewhere between 2003 and 2004, I was having a discussion over AIM with my friend Ryan. We had the incredibly clever idea of making screen names that literally told people who we were. This dawned my previous screen name and Ryan's current screen name, its me barak and itsa me uh ryan. Again, I was not satisfied, something was missing from my screen name. Finally, a few days later, it hit me, its barak yo would be my screen name, who wouldn't like that?!

Birth of Barakyo

With the creation of my screen name, Barakyo was born! It wasn't until later on in high school that my nickname really started catching on. It seemed my screen name grew popular amongst my friends as whenever they would see me, they'd always say something along the lines of, "Hey! It's Barak, yo." Somehow through the recitement of my screen name, the comma was dropped and my name just became, Barakyo.

In addition to my popular screen name, my away messages, which were stolen from popular rap songs, seem to also aid my nickname. Some of my more catchy away messages included:

It's Barakyo commin' out yo stereo

and

But I ain't 5-0, yall know its Barakyo!

Barakyo today

Present day Barakyo is a simpler man. One who does not refer to himself in the third person :P I often still get phone calls where the minute I answer the phone the person on the other line will yell, "Barakyo comin' out my stereo!" I love these phone calls, they put a smile on my face the second I hear them. In addition, I still use my nickname Barakyo whenever something calls for a more personalized touch, hence my blog :)

Now you know all there is to know about Barakyo...


Writing Test Cases for Laravel

10 Nov 2013

Besides writing test cases just because "everyone else is", we should be writing test cases since they'll help create a stronger trust in our code that our functions are doing what they should be doing.

Requirements

  • PHP
  • PHPUnit (you will most likely need a global install)
  • Test Cases (DUH)

Helpers

Helpers are a set of classes that we create to take care of common mundane tasks. For example, the User Helper for testing provides functions for returning a user with a specified role or easily create a new user with the specified role if one does not exist or the user requests one.

Since we can define helpers for a number of things, we want to ensure that the helpers for our test case don't conflict with helpers that we use in other applications, for this we'll create our test helpers in the sub-namespace Testing within the Helpers name space.

PHP Namespaces are defined as the first line after the <?php opening. It MUST be the first line after the opening, or else PHP will throw an error.

<?php

namespace Helpers\Testing;

After defining the namespace, you will most likely want to include common exceptions that will be thrown when accessing models.

use Illuminate\Database\Eloquent\ModelNotFoundException;

From there, you're ready and free to define your helper class as you please.

PLEASE COMMENT FUNCTIONS

Test Cases

To write test cases you must extend from the Laravel's base TestCase class. When writing test cases please follow certain conventions:

  • The name of the file and the class for the test case should follow the naming convention ControllerFunctionTest
    • Ex: Writing test cases for the Index function for the User Controller: UserControllerIndexTest.php
<?php
class UserControllerIndexTest extends TestCase {

}
  • All functions that are test musted start with the word test (PHPUnit convention) or else PHPUnit will not find your function
  • Functions should specify what role they're testing as, ex: testIndexAsAdmin()

Tips for Writing Test Cases

Calling Routes

We want to guarantee our that not only our functions are correct, but that our routes/API are calling the correct functions. If you can, please test routes rather than functions. You can call routes using the $this->call() function. More information for calling routes can be found in Laravel's Documentation.

Acting as a User

To act as a user, you can use the $this->be() function, which can also be found in Laravel's Documentation

Specifying Exceptons

An exception being throw can be a success if a user should not have access to certain functionality. Since our API immediately throws a InsufficientPermissionsException when a user doesn't have the right permissions, we can easily have our test cases catch this and mark them as a passed test. To specify that an exception should be thrown, you can define it in the PHPDocs comments like so:

<?php
    class UserControllerIndexTest extends TestCase {
        /**
         * Tests user index function as student, passes on thrown exception
         * @expectedException InsufficientPermissionsException
        */
        public function testIndexAsStudent() {
            $user = $this->userHelper->getStudentUser();
            $this->be($user);
            $this->call("GET", $this->base_route);
        }
    }

Writing Functions

Note: Only one test per function, so you cannot have a function return users with all roles and test them as one test case. Hence the need for the naming convention of what you're testing as what role.

Test cases can become repetitive, to ease the amount of typing you can write a function (GASP!) which we can call in our all most of our tests.

References


Weekly Report 10/13-10/19

21 Oct 2013

Good:

  • Team meeting with good discussions and thoughts
  • Completed usability requests (Thanks DUDE!)
  • Discussions with various people regarding my thesis and some ideas
  • Was able to find an answer to one of the problems I was having with my thesis
  • Swam 4 times and swam my fastest 50m so far! (35 seconds)
  • Big picture discussions about web services, API keys, improving the API
  • Completed and demoed my project for Comp595.
  • Began the path to more focused test driven development in our latest sprint planning
  • Planning Poker at the sprint planning
Areas for Improvement:
  • More efficient planning
  • Prioritization and organization
  • Patience
    • Hard work does not go unnoticed (I hope!)
Thoughts:
  • Made some big decisions over the weekend testing my ability to be more decisive and I think I'm doing well.


Weekly Report 9/15-9/21

22 Sep 2013

My boss emails out a weekly report of all his activities every week and I thoroughly enjoy them. I'd thought I'd start writing them myself to help reflect. Good:

  • Completed notifications for RP
  • Started working on a small email wrapper class for RP
  • Progress on my thesis
  • Completed section on Phoenix database's layer.
  • Completed section on the Neo4J equivalent of EAS's database.
  • Swam 4 times this past week!
  • Completed and submitted Comp429's Project 1 - Yak
  • Completed, submitted, and demoed Comp595's Project 2 - Sensor Information
  • Discussions with coworkers
    • Compilers, Interpreters, JIT compilers
    • C
      • Pointers/Arrays and Buffers
      • Re-entrant functions
      • Man pages
  • RP pre-meeting and role reviews
  • Revised/updated my resume
  • Got RP running on saga28
  • Will be helping out with my friend's robotics team at his old high school, Team4Element!
  • Completed 2 Project Euler problems in Python!
Areas for Improvement:
  • Communicating efficiently
  • Saga28 server maintenance
    • Messed up existing SSL configuration when updating Apache :/
  • Need to get back on my scheduled 1-on-1s
  • Have to get back to updating my blog
  • Organization, scheduling, and Agile
  • Lost touch of the Agile approach since I've moved back to RP.
  • Thesis proof-reading and progress updates with my thesis committee.
  • Wasn't so active on IRC this past week.
Thoughts:
  • Team Building
    • Team4Element - Need to learn the kids and how I can help
    • Family - Need to figure out how to help out here as well
  • Python-Eve Neo4J driver
    • Reviewed what I could do to possibly develop a Neo4J driver for the Eve framework built on top of Flask.


I've been making the switch to tiling window managers through the use of AwesomeWM. If you're not using a tiling window manager, you should definitely give it a shot! I shouldn't take all the credit though, Copycat Killer's AwesomeWM configs are a great start and essentially what allowed me to make the switch.


Python-Excel: XLRD & XLWT

29 May 2013

The other week we received a request for a small script from a department on campus. The department needed to evaluate data from a Microsoft Excel worksheet which they wrote a macro for. Their problem, though, was to aggregate all their data into one sheet. This aggregation process was not only the most tedious but was also their longest task taking nearly 2 weeks to complete. My coworkers and I were shocked that they've continued this process for so long without looking to automate it somehow, so we took on the challenge to ease the pain in their lives.

I set out to write a Python script and immediately researched what Microsoft Excel libraries are available for Python. I quick found Python-Excel and the XLRD and XLWT libraries, which I have to say, have been nothing short but amazing to use. The libraries are intuitive and easy to use. There was not a function I could think of that this library didn't provide. The code can be found on GitHub.

Just some quick examples of how I was using the libraries:

Opening an excel file:

try:
    # Open the book
    book = xlrd.open_workbook(file_name)
except IOError:
    logging.warning("File: " + file_name + " does not exist")

Opening a specific sheet:

# Try to open the specific sheet
try:
    summary_sheet = book.sheet_by_name('For Summary File')
except:
    logging.warning('Could not open sheet ' + sheet_name + ' in file ' + file_name)

Copying a row:

row = [summary_sheet.cell(1, col).value for col in range(summary_sheet.ncols)]

Creating a new workbook and sheet:

# Create a new workbook
new_workbook = xlwt.Workbook()
# Create a new sheet
new_sheet = new_workbook.add_sheet('Sheet1')

Writing data to the new workbook:

# Write the data to the sheet
new_sheet.write(row, col, label=current_row[col])

Saving a workbook:

new_workbook.save(output_file)

Doesn't get much simpler than that! :)


It seemed to be happening more often than usual. I'd have something cool to share but no one was on messenger/IRC/Facebook/Google+/etc. What good are all these social networks if its so difficult to just share the latest JS plugin I saw on Reddit/Github? Going back over my options, I thought maybe I could just drop my friends an email, but the more I thought about it, emails can be annoying and are typically dismissed. If I'm going to take the time to share something, I want someone to see it! I needed a good way to be able to share links, update those that want to see it without annoying them, and have these links be accessible anywhere. In steps Kippt. Kippt is social bookmarking taken to the next level. My coworkers and I can now share links easily right through a browser extension and can be notified based on how we want. We've been using Kippt for about a month and a half now and its been an integral part of the way we work, learn, and share.

If you're on Kippt, follow me and I'll follow you back: Barakyo

If you're not on Kippt, I highly recommend getting on, following people (and me if you want!)


My first jQuery Plugin--SelectAll

03 Feb 2013

Refactoring code has been on my mind more lately than usual. I feel that if I'm not actively thinking and keeping in mind of what I'm coding, that I'm bound to rewrite code I've already written elsewhere. While working on Gavel's "Manage" page (still thinking of a clever title for that page), I realized that I had a set of checkboxes that I needed to keep track of for both my labels tab and my events tab. I wrote some small Javascript code to handle keeping track of labels as well as select/deselect all checkboxes, but when I started the events section, I realize that I didn't want to have to rewrite all this code. A lightbulb went off in my head and rather than googling for a jQuery plugin, I decided to challenge myself and write my own, so I present jQuery Select All. The plugin will take checkboxes as an input and keep track of them in an array. An optional select/deselect all can be passed in with the allSelector parameter. I'm sure this has been written already, but it was a good challenge for myself to think abstractly. Go check it out and fork me if you want!


Love this commercial and the feel good mood!