Play Framework: Returning JSON Responses

06 Apr 2014

I've recently started a project at working using the Play Framework and while its a great framework, I was having a lot of trouble with some of the simplest tasks. I wouldn't blame Play for my problems, returning to Java after a long hiatus, being spoiled by dynamicly typed languages, and lack of documentation really made such tasks like returning a JSON response difficult.

I figured that I may not be the only in this position, judging by the questions in the IRC channel and lack of responses, I figured it may be a good idea to jot down some of my notes, not only for myself, my coworkers, but for all my fellow Play framework noobs. Apologies for the introduction, I'll try to keep the rest of this posts and the futures, short, succinct, and to the point.

tl;dr

Use ObjectNode class found in the com.fasterxml.jackson.databind.node.ObjectNode package.

import com.fasterxml.jackson.databind.node.ObjectNode;

    ObjectNode response = Json.newObject();
    response.put("status", 200);
    response.put("message", "Request was successful");
    response.put("data", Json.toJson(users));
    return ok(response);

Using ObjectNode

To easily return mixed response types, I suggest using the ObjectNode class found in the com.fasterxml.jackson.databind.node.ObjectNode package. Using the class is simple, first instantiate an object:

ObjectNode response = Json.newObject();

From there, you can treat the object similar to a map but using put() function.

response.put("status", 200);
    response.put("message", "Request was successful");

Often times we want to return an object or array of objects. Using the Json.toJson() function found in play.libs.Json we can easily convert objects or lists of objects to be included in our JSON response.

response.put("data", Json.toJson(users));

Finally, you'll probably want to use the static return response methods to easily return our JSON response.

return ok(response);


Play Framework: Parsing Arrays in JSON Requests

06 Apr 2014

tl;dr

I suggest using ObjectMapper found in the com.fasterxml.jackson.databind.ObjectMapper

ObjectMapper mapper = new ObjectMapper();
    try {
        JsonNode root = request().body().asJson();
        JsonNode jsonPhoneNumbers = root.get("phoneNumbers");
        for(JsonNode phoneNumberString : jsonPhoneNumbers) {
            String phoneNumber = phoneNumberString.asText();
        }
    catch (NullPointerException npe) {
        // No valid phone numbers were provided
    }

Using ObjectMapper

For parsing arrays in JSON requests, I suggest using the ObjectMapper. Assuming you have a request that looks like:

{
        phoneNumbers: [
            '555-555-5555',
            '555-555-5556',
            '555-555-5557'
        ]
    }

We can easily access the phone number values by using ObjectMapper class. The first thing that you'd want to do is instantiate an instance of this class, which is pretty straight forward.

ObjectMapper mapper = new ObjectMapper();

From there, we want a JSON version of our request, the Play framework allows you to easily do so by chaining the asJson() method to the request().body() functions.

JsonNode root = request().body().asJson();

Using the get() function on our root object and providing the correct key, we can easily get our phoneNumbers as a JsonNode, which is also iterable. By iterating through each of the JsonNode objects, we can just return the value for the corresponding phone number as a String by using the asText() method.

JsonNode jsonPhoneNumbers = root.get("phoneNumbers");
    for(JsonNode phoneNumberString : jsonPhoneNumbers) {
        String phoneNumber = phoneNumberString.asText();
    }

Since there is a possibility that the request could be empty (null), I suggest wrapping the request processing in a try-catch block. If a request is null or the specified key was not included in the request (in this example it would be phoneNumbers) then we can use the catch portion of the block to easily return a bad request.

Put this all together looks like:

import com.fasterxml.jackson.databind.JsonNode;
    import com.fasterxml.jackson.databind.ObjectMapper;
    import com.fasterxml.jackson.databind.node.ObjectNode;
    import play.libs.Json;

    ObjectMapper mapper = new ObjectMapper();
    ObjectNode response = Json.newObject();
    try {
        JsonNode root = request().body().asJson();
        JsonNode jsonPhoneNumbers = root.get("phoneNumbers");
        for(JsonNode phoneNumberString : jsonPhoneNumbers) {
            String phoneNumber = phoneNumberString.asText();
        }
    catch (NullPointerException npe) {
        // No valid phone numbers were provided
        response.put("status", 400);
        response.put("message", "No phone numbers were provided");
        return badRequest(response);
    }


More TDD and Recursion Practice

24 Nov 2013

It seems that Javascript challenges are more frequent at work. Thursday afternoon when we got back from lunch, my coworker was testing himself in the You Can't Javascript Under Pressure challenges. As great coworkers, we (by that I mean, me) decided to help out and put more pressure on our fellow coworker, without even being asked to! One of the challenges that came up during the test was a function ArraySum() which accepts a list (array) of values and you must total all the integers within the array. The array could contain strings, numbers, booleans, lists of strings, numbers, booleans, etc. Regardless of the input, you must total all the integers within the array. After seeing that challenge (and helping with the correct answer), I thought it might be good practice for another blog post.

Practice makes Perfect

Before jumping into more testing and programming, I want to stop and think about my methodology. While re-reading Chapter 3 of Test-Driven Web Development with Python on Friday with a coworker, I realized that almost every line of code that was written in the chapter to satisfy the test, had a test case as a reason to write that code. After seeing this and thinking about my previous blog post, I realized I didn't follow the TDD methodology as closely as I couldshould have. According to the author, Harry Percival, TDD is a discipline, it even says so in chapter 4.

TDD is a discipline, and that means it's not something that comes naturally; because many of the payoffs aren't immediate but only come in the longer term, you have to force yourself to do it in the moment.

To cut myself some slack, TDD is something you have to practice. Yes, I'm talking about practice. I'm sure there are a bunch of young grasshoper learning quotes I could state right now, but lets get on to testing.

Stricter Test Cases

Last time we started by importing our function and writing a basic test case to test the return type. Let's do that again.

import unittest
from arraysum import ArraySum

class ArraySumTests(unittest.TestCase):
    def setUp(self):
        pass

    def test_for_int(self):
        result = ArraySum([1, 2, 3, 4, 5,])
        self.assertTrue(type(result) is int, "Result is not integer")

if __name__ == '__main__':
    unittest.main()

Looks good to me, lets run it!

Traceback (most recent call last):
File "tests.py", line 2, in <module>
  from arraysum import ArraySum
  ImportError: No module named arraysum

Our first error! Lets correct it, but remember, minimal amount of code! In fact, we won't right any code at all.

caster:arraysum/ $ touch arraysum.py

Let's re-run our test.

Traceback (most recent call last):
File "tests.py", line 2, in <module>
  from arraysum import ArraySum
  ImportError: cannot import name ArraySum

Yay new error! This time it can't find our function, ArraySum, lets (minimally) create that in our new arraysum.py file.

def ArraySum(int_list):
    pass

Okay, let's see what we get now.

F
======================================================================
FAIL: test_for_int (__main__.ArraySumTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests.py", line 10, in test_for_int
  self.assertTrue(type(result) is int, "Result is not integer")
  AssertionError: Result is not integer

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (failures=1)

MoreMOAR errors!! This time our test is failing because we're not returning an integer. Let's fix that by modifying our return statement.

def ArraySum(int_list):
    return 0

Alright, lets re-run our test.

.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Woohoo! Our first test passed. According to TDD, we can now write some more tests. Let's make this one a little more interesting.

import unittest
from arraysum import ArraySum

class ArraySumTests(unittest.TestCase):
    def setUp(self):
        pass

    def test_for_int(self):
        result = ArraySum([1, 2, 3, 4, 5,])
        self.assertTrue(type(result) is int, "Result is not integer")

    def test_flat_list(self):
        result = ArraySum([1, 2, 3, 4, 5])
        self.assertTrue(result == 15, "Incorrect sum")

if __name__ == '__main__':
    unittest.main()

We've now added a new test, test_flat_list to test a simple list of integers and ensure the sum is correct. Back to running tests...

F.
======================================================================
FAIL: test_flat_list (__main__.ArraySumTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 14, in test_flat_list
  self.assertTrue(result == 15, "Incorrect sum")
  AssertionError: Incorrect sum

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (failures=1)

The obvious problem here is that our function is returning 0 and not doing any sort of summation. According to TDD, we want to write the minimal amount of code, Python, being the beautiful language that it is, provides us with a simple sum function which operates on iterables. Allowing us to very easily pass our test_with_flat_list test.

def ArraySum(int_list):
    return sum(int_list)

Running our test again will show that we were able to successfully pass it.

..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

We can continue writing more test cases, so lets make it a little more interesting. We'll introduce some non-integer values into the list, which will create errors for our sum function in a test case called test_complex_list.

import unittest
from arraysum import ArraySum

class ArraySumTests(unittest.TestCase):
    def setUp(self):
        pass

    def test_for_int(self):
        result = ArraySum([1, 2, 3, 4, 5,])
        self.assertTrue(type(result) is int, "Result is not integer")

    def test_flat_list(self):
        result = ArraySum([1, 2, 3, 4, 5])
        self.assertTrue(result == 15, "Incorrect sum")
    
    def test_complex_list(self):
        result = ArraySum([1, 2, 3, "hello"])
        self.assertTrue(result == 6, "Incorrect sum")


if __name__ == '__main__':
    unittest.main()

Running our tests, returns the results:

E..
======================================================================
ERROR: test_complex_list (__main__.ArraySumTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests.py", line 17, in test_complex_list
  result = ArraySum([1, 2, 3, "hello"])
    File "/home/caster/Development/arraysum/arraysum.py", line 2, in ArraySum
        return sum(int_list)
        TypeError: unsupported operand type(s) for +: 'int' and 'str'

----------------------------------------------------------------------
Ran 3 tests in 0.000s

FAILED (errors=1)

Just as we expected! Let's make some corrections to our ArraySum function.

def ArraySum(int_list):
    sum = 0
    for x in int_list:
        if type(x) is int:
            sum += x
    return sum

After making our fix, we'll re-run our test and...

...
----------------------------------------------------------------------
Ran 3 tests in 0.000s

OK

Looks good! Once again, passed test cases means we can write more tests, so lets give it a go one last time. We'll create a new test case, test_complex_nested_list, where we'll introduce a nested list into our previous complex list. A nested list within our function also brings in the sense of recursion! We'll want to take in account what we have, where we want to go, and if we've done something like this before (which we have). Currently we're iterating through all the values and if they're integers we'll add them to our sum variable. We're throwing away that is anything but an int. We know this is wrong though since the function must also include the values within nested lists, hence the embarrassingly recursive algorithm.

We know that we're going to have to iterate through the values in our list, so we understand that, but lets look at our cases:

  • Value is an int: Add to sum
  • Value is a list: Add the total of the list to sum
  • Value is neither: Ignore

Now that we have an idea of what we want to do, lets write our last test case.

import unittest
from arraysum import ArraySum

class ArraySumTests(unittest.TestCase):
    def setUp(self):
        pass

    def test_for_int(self):
        result = ArraySum([1, 2, 3, 4, 5,])
        self.assertTrue(type(result) is int, "Result is not integer")

    def test_flat_list(self):
        result = ArraySum([1, 2, 3, 4, 5])
        self.assertTrue(result == 15, "Incorrect sum")
    
    def test_complex_list(self):
        result = ArraySum([1, 2, 3, "hello"])
        self.assertTrue(result == 6, "Incorrect sum")

    def test_complex_nested_list(self):
        result = ArraySum([1, 2, 3, "hello", [4, 5]])
        self.assertTrue(result == 15, "Incorrect sum")

if __name__ == '__main__':
    unittest.main()

Before we make any changes to ArraySum, we have to run our test.

.F..
======================================================================
FAIL: test_complex_nested_list (__main__.ArraySumTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests.py", line 22, in test_complex_nested_list
      self.assertTrue(result == 15, "Incorrect sum")
      AssertionError: Incorrect sum

----------------------------------------------------------------------
Ran 4 tests in 0.000s

FAILED (failures=1)

Now that we have a failed test, we are given the okay by the testing goat to modify our code. Let's add our recursive call, so our function now looks like:

def ArraySum(int_list):
    sum = 0
    for x in int_list:
        if type(x) is int:
            sum += x
        elif type(x) is list:
            sum += ArraySum(x)
    return sum

Re-running our final test...

....
----------------------------------------------------------------------
Ran 4 tests in 0.000s

OK

Ahhhhh :)


Code Challenge: Dictionary Flatten

17 Nov 2013

While browsing /r/javascript Thursday night, I came across a post regrading a mildly interesting interview question which had to do with flattening a Javascript object to a URI.

Two things immediately jumped into my mind:

  1. I have to show one of my coworkers.
  2. This would make a "midly interesting" Python challenge!

Friday morning, the first thing I did when I got into work was write the challenge on our white board and my solution in Python. Funny enough, once I came back from lunch, my coworker was writing his Javascript solution next to my Python one. It turns out that he saw the challenge Thursday night and even posted his solution to the thread.

I figured, I'd code up my solution and give it a shot. Both for recursive practice and writing test cases. So to start with some test driven development (TDD), I began by writing a simple test case.

First we'll define a class for our test cases:

import unittest
class FlattenTests(unittest.TestCase):

From there, we'll add a simple setUp method which creates a dictionary to use for testing:

import unittest

class FlattenTests(unittest.TestCase):
    def setUp(self):
        self.example = {
                "foo": {
                    "bar": {
                        "team": True,
                        "company": ["Bill", "Ted"]
                        }
                    }
                }

Now lets define a test case!

import unittest

class FlattenTests(unittest.TestCase):
    def setUp(self):
        self.example = {
                "foo": {
                    "bar": {
                        "team": True,
                        "company": ["Bill", "Ted"]
                        }
                    }
                }

    def test_for_dict(self):
        d = flatten(self.example, "")
        self.assertTrue(type(d) is dict)


if __name__ == '__main__':
    unittest.main()

The test case we've defined, test_for_dict, just determines if the return type of the flatten method (which we haven't written yet) is of type dictionary. Lets run our test case and see what we get!

ERROR: test_for_dict (__main__.FlattenTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 16, in test_for_dict
d = flatten(self.example, "")
NameError: global name 'flatten' is not defined
----------------------------------------------------------------------

Ahh what was that!? According to my understanding of TDD, this is a good thing! We can start writing code now. Lets define a new file flatten.py and create a flatten function that accepts a dictionary and string. Remember, according to TDD, we must right the most minimal amount of code that will pass our test.

def flatten(d_flat, key_string):
    pass

Well that was simple enough, lets run our test.

======================================================================
FAIL: test_for_dict (__main__.FlattenTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 17, in test_for_dict
self.assertTrue(type(d) is dict)
AssertionError: False is not true
----------------------------------------------------------------------

Hmm, seems like we didn't write enough code! Lets go back and update our flatten function.

def flatten(d_flat, key_string):
   return {}

Now that we've updated our code, lets run our test again.

caster:flatten/ $ python tests.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

Looks good to me! Now that one of our tests has passed, we can now write another (more interesting) test! I'm not very creative at writing these incremental tests, so lets just get to it.

import unittest
from flatten import flatten

class FlattenTests(unittest.TestCase):
    def setUp(self):
        self.example = {
                "foo": {
                    "bar": {
                        "team": True,
                        "company": ["Bill", "Ted"]
                        }
                    }
                }

    def test_for_dict(self):
        d = flatten(self.example, "")
        self.assertTrue(type(d) is dict)

    def test_case_one(self):
        d = flatten(self.example, "")
        self.assertTrue(d["foo/bar/team"])
        self.assertTrue(d["foo/bar/company"] == ["Bill", "Ted"]) 

if __name__ == '__main__':
    import unittest

We've defined a new test case, test_case_one, (like I said, I'm not very creative), that'll run our flatten function and determine if we've received the correct output. Lets try running our test case now (and cross our fingers) to see if it fails.

caster:flatten/ $ python tests.py
E.
======================================================================
ERROR: test_case_one (__main__.FlattenTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 21, in test_case_one
self.assertTrue(d["foo/bar/team"])
KeyError: 'foo/bar/team'

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (errors=1)

Awesome! More good news! :) Before we jump into writing our function, lets think about what we want the function to do.

  • First we'll need to iterate through all the key/value pairs in the dictionary passed to the function.
  • We'll then need to determine if the value for the key we've iterated to is a dictionary or not.
    • If the value is a dictionary, we'll want to recursively call flatten and append our key to key_string so that we're keeping track of the key we want to eventually use.
  • The easy (or base) case, if our value is NOT a dictionary, simply add it to the dictionary with it's key append to key_string
  • Finally, once we've iterated through all the values, return the dictionary.

When we put it like that, it sounds easy! (Or so I hope...)

Lets take care of iterating through the dictionary, that sounds easy enough:

def flatten(d_flat, key_string):
    d = {}
    for k,v in d_flat.items():
        pass
    return d

Simple enough, we've declared an empty dictionary d and started iterating through the items in the dictionary. Lets take a look at our first case, if the value for a key is a dictionary:

def flatten(d_flat, key_string):
    d = {}
    for k,v in d_flat.items():
        if(type(v) is dict):
            pass
        else:
            pass
    return d

Okay we've added our if statement, now if it's true, we'll have to recursively call our function. Before we jump to make the recursive call, we need to ensure we're keeping track of our key_string value. To do so, we'll pass key_string with the current key value, k. We'll recursively call flatten with the inner dictionary stored in v.

def flatten(d_flat, key_string):
    d = {}
    for k,v in d_flat.items():
        if(type(v) is dict):
            d = flatten(v, key_string + k + "/")
        else:
            pass
    return d

Almost there! We now need to worry about our base case, so let's take care of that! Recall that we said if our value is not of type dictionary, we simple append it to our dictionary that we declared, d. Again, we have to keep in mind that we want to assign the correct key for our value, which would be our current key_string concatenated with the current key we're looking at, k. Once we create the right key, we'll just assign it to our value, v and return our dictionary, d.

def flatten(d_flat, key_string):
    d = {}
    for k,v in d_flat.items():
        if(type(v) is dict):
            d = flatten(v, key_string + k + "/")
        else:
            d[key_string + k] = v
    return d

So good so far, lets run our tests!

caster:flatten/ $ python tests.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

Perfect...or so we think. There's actually a problem in our algorithm, which we can expose by making our testing data little more robust. Lets update our setUp and our test_case_one functions so they look like this:

import unittest
from flatten import flatten

class FlattenTests(unittest.TestCase):
    def setUp(self):
        self.example = {
                "foo": {
                    "bar": {
                        "team": True,
                        "company": ["Bill", "Ted"]
                        }
                    },
                    "fizz": {
                        "blue": "red"
                    }
                }

    def test_for_dict(self):
        d = flatten(self.example, "")
        self.assertTrue(type(d) is dict)

    def test_case_one(self):
        d = flatten(self.example, "")
        self.assertTrue(d["foo/bar/team"])
        self.assertTrue(d["foo/bar/company"] == ["Bill", "Ted"]) 
        self.assertTrue(d["foo/bar/fiz/blue"] is "red")

if __name__ == '__main__':
    unittest.main()

We've added another key that points to another dictionary. Nothing unusual or out of the oridinary. We've also added a new assertion to ensure that the key foo/bar/fiz/blue contains the string red. Lets run our test case and see what happens.

caster:flatten/ $ python tests.py
E.
======================================================================
ERROR: test_case_one (__main__.FlattenTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests.py", line 21, in test_case_one
self.assertTrue(d["foo/bar/team"])
KeyError: 'foo/bar/team'

----------------------------------------------------------------------
Ran 2 tests in 0.000s

FAILED (errors=1)

Failed! What's going on here? If you take a look at our algorithm, we're not updating our dictionary when we return from a recursive call, we're actually overwriting it. Let's make a small change and ensure we're updating our dictionary and see what happens.

def flatten(d_flat, key_string):
    d = {}
    for k,v in d_flat.items():
        if(type(v) is dict):
            d.update(flatten(v, key_string + k + "/"))
        else:
            d[key_string + k] = v
    return d

Now lets run our tests.

caster:flatten/ $ python tests.py
..
----------------------------------------------------------------------
Ran 2 tests in 0.000s

OK

All is well and now our function works as expected!


Writing Test Cases for Laravel

10 Nov 2013

Besides writing test cases just because "everyone else is", we should be writing test cases since they'll help create a stronger trust in our code that our functions are doing what they should be doing.

Requirements

  • PHP
  • PHPUnit (you will most likely need a global install)
  • Test Cases (DUH)

Helpers

Helpers are a set of classes that we create to take care of common mundane tasks. For example, the User Helper for testing provides functions for returning a user with a specified role or easily create a new user with the specified role if one does not exist or the user requests one.

Since we can define helpers for a number of things, we want to ensure that the helpers for our test case don't conflict with helpers that we use in other applications, for this we'll create our test helpers in the sub-namespace Testing within the Helpers name space.

PHP Namespaces are defined as the first line after the <?php opening. It MUST be the first line after the opening, or else PHP will throw an error.

<?php

namespace Helpers\Testing;

After defining the namespace, you will most likely want to include common exceptions that will be thrown when accessing models.

use Illuminate\Database\Eloquent\ModelNotFoundException;

From there, you're ready and free to define your helper class as you please.

PLEASE COMMENT FUNCTIONS

Test Cases

To write test cases you must extend from the Laravel's base TestCase class. When writing test cases please follow certain conventions:

  • The name of the file and the class for the test case should follow the naming convention ControllerFunctionTest
    • Ex: Writing test cases for the Index function for the User Controller: UserControllerIndexTest.php
<?php
class UserControllerIndexTest extends TestCase {

}
  • All functions that are test musted start with the word test (PHPUnit convention) or else PHPUnit will not find your function
  • Functions should specify what role they're testing as, ex: testIndexAsAdmin()

Tips for Writing Test Cases

Calling Routes

We want to guarantee our that not only our functions are correct, but that our routes/API are calling the correct functions. If you can, please test routes rather than functions. You can call routes using the $this->call() function. More information for calling routes can be found in Laravel's Documentation.

Acting as a User

To act as a user, you can use the $this->be() function, which can also be found in Laravel's Documentation

Specifying Exceptons

An exception being throw can be a success if a user should not have access to certain functionality. Since our API immediately throws a InsufficientPermissionsException when a user doesn't have the right permissions, we can easily have our test cases catch this and mark them as a passed test. To specify that an exception should be thrown, you can define it in the PHPDocs comments like so:

<?php
    class UserControllerIndexTest extends TestCase {
        /**
         * Tests user index function as student, passes on thrown exception
         * @expectedException InsufficientPermissionsException
        */
        public function testIndexAsStudent() {
            $user = $this->userHelper->getStudentUser();
            $this->be($user);
            $this->call("GET", $this->base_route);
        }
    }

Writing Functions

Note: Only one test per function, so you cannot have a function return users with all roles and test them as one test case. Hence the need for the naming convention of what you're testing as what role.

Test cases can become repetitive, to ease the amount of typing you can write a function (GASP!) which we can call in our all most of our tests.

References