How to Test File Upload Parsing in Django

I didn’t start testing software until recently. Of course I tested my software to make sure it works and is stable, but it was a very manual process. This was very time consuming and sometimes subtle bugs were let through or re-introduced in large codebase changes.

I didn’t start unit-testing for a long time because I didn’t really know where to begin. Especially with an existing codebase. I’ve been working on a project and part of the project includes a file upload that needs to be parsed. I couldn’t stomach the thought of clicking, browsing, submitting and waiting each time just to test a few changes.

Problem

Testing uploaded file parsing in a manual fashion is very time consuming and takes a long time. If the parsing breaks, we won’t know unless we upload each test file after every change. We can do better than this.

We need to utilize the testing framework in Django to help us automate this process a bit more.

Solution

The first thing we want to do is make an example file that a user may upload in our fixtures directory. Keeping the test upload files in our fixtures has a number of advantages. It allows us to keep them in our version control and lets all of our testing be portable.

For the example let’s pretend we are parsing an excel file of some kind.

The next step is to prepare our test. While we are only testing a single file upload, you may want to add in corrupted files or perhaps expand to different versions. So, lets setup our test with a single test “runner” test that will run the tests.

Also note that by keeping the import statements using a localized path instead of a full package sitename.app.file, we can further improve portability.

from django.test import TestCase
from utils import parse_excel_upload
import os

class ExcelParseTest(TestCase):
    
    test_upload_path = 'fixtures/test_upload.xls'
    
    def test_parse(self):
      self.parse(self.test_upload_path)

The above is straight forward, we have our excel test parse class, and a method named test_parse that will call a method named parse that takes a filename and will run all the actual tests.

Next, let’s setup our actual parse method:

 def parse(self,file_location):
        """
        Tests that we can parse our excel files
        """
        
        #setup
        path = os.path.join(os.path.split(__file__)[0],file_location)

        elements = parse_excel_upload(path)
        
        #Assert that we have something returned as expected
        self.assertNotEqual(elements,None)
        # continue doing more specific tests here

Note that I am using self.assertNotEqual. If you are running Python 2.7 or later, you can use self.assertNotNone(elements) for the same functionality. Using assertNotEqual simply allows us to run a older versions of python as well.

If we wanted to test our method with more files, perhaps some corrupted files, we simply need to add another test file in our fixture and then modify our runner to include parse the new file.

Conclusion

Testing requires a small initial time investment to get started. If it’s your first time testing, there are a few mental gymnastics to get used to for writing code that is testable.

After the initial investment, which becomes less and less each time do it, we can vastly improve the reliability of our code and catch subtle bugs that may crop up during development or future releases.