New application backend!

After writing my initial "Prototype" backend (which is still running in 'read-only' mode) I was faced with having to maintain the service and fix bugs, and let's not forget adding new features as needed. As with all prototypes, there were some peices that were sort of smashed together just to make the system work with minimal effort. Security was thrown out the window, to reduce the time to develop the core features, and was developed in NodeJS because they have PEG.js

To say the least, I was satisfied with the outcome of the prototype however unhappy with the quality of the code. After a couple days of trying to refactor the Javascript on the server, I had an epiphany. I don't want Javascript running on my server as the core service of this platform I am creating. It was to complicated to take a 'Prototype' and make it production ready. I was finding all sorts of issues with Closures, and trying to rafactor code that was nested 5 layers deep in callbacks. I know that NodeJS now has Promises, however not all of the libraries I was using support Promises yet. At the time MongoDB specifically did not, and that was a deal breaker for me. Here comes Python!

After doing some testing and research on Parsing Libraries in other languages, I finally decided on Python. I know that it doesn't benchmark as fast as NodeJS for computational speed, and that it is not "All the rage!" with the new web developers. It does however, have: Many libraries, A huge community, Stable language features, Blocking and non-blocking I/O, Generators, and many more good attributes. After I started the rewrite, I quickly realized why so many people that use python absolutely love it, Stable Libraries.

Libraries Used

There were many features in the NodeJS version of the service that I was expecting to have to write myself, because I either had to in NodeJS or I figured that Python probably wouldn't have it, because NodeJS had just barely gotten the library. To my surprise, I was wrong. I ended up not having to write a single library in python (Other than my custom DB layer for business logic). Python has via PyPi all of the libraries I wanted and more:

  • MessageQueue: Pika
  • MongoDB Connector: PyMongo
  • Text Parser: Arpeggio
  • Google Service API: gcloud
  • Framework: Flask, Flask-JWT

Simplistic Design

One of the main benefits that I have enjoyed after this rewrite using Python instead of NodeJS is simplicity. The design of the Service (you might call it a microservice) which handles basic content, is much simpler now. I was able to easily break out more of the logic into seperate functions, and use less nesting, objects, and so on. The majority of the new code stays close to the left and is easily composible for new functionality.

For example here is the get sites route in Python ('/server.py'):

@app.route('/api/v1/sites', methods=['GET'])
@jwt_required()
def route_get_sites():
    user = current_identity['user']
    sites = saltmine.get_sites(user)

    return JsonResponse(sites)

Notice that I didn't need to change a routes file, or add it to the "Object" of a route. I am simply able to create a Function, and add a Decorator to it for the route, as well as requiring a valid Token in the request. Hopefully you see how this makes developing API's easier. Not only initially, but I suspect that it will be much easier to maintain this code.

NodeJS for reference:

...
// /routes/index.js
router.get('/api/v1/sites', sites.getSites);
...

...
// /routes/sites.js
getSites: function(req, res) {
	//var site_id = req.

	var match = {};
	var items = [];

	var domains = [];

	for (var permission in req.token.user.roles) {
		for (var site_id in req.token.user.roles[permission]) {
			domains.push(new mongo.ObjectId(req.token.user.roles[permission][site_id]));
		}
	}

	var cursor = req.db.collection('sites').find({_id: {$in: domains}});
	cursor.each(function(err, doc) {
		console.log(doc);
		if (doc != null) {
			items.push(doc);
		} else {
			res.json(items);
		}
	});
},
...

Any Performance Loss?

I have to admit, that there has been no noticeable performance difference. I was fully expecting Python to be a little slower than NodeJS. Especially considering the benchmarks I have seen around the internet. But after testing the system out, and trying to find the bottlenecks, I figured out that MongoDB is actually where most of the resources and time are spent. Basically the Python and Javascpript both receive a request, massage the data and request stuff from the Database; sit around, while the database processes the request and hands the data back. The amount of time spent actually running in Python or Javascript is so little, that it doesn't make a difference in the grand scheme of things.

My Conclusion!

After this experience, it will be hard to ever bring myself to willingfully code in Javascript outside of the browser's client. I will try my best to create my new projects in Python where it fits. I find myself more productive in Python and happier not trying to debug callback stack traces. I still do a lot of work in PHP (at ProSoft Technology), C & C++ for some performance demanding projects, and other languages here and there for fun (including my own Gamma Language). What I can say is that I will not start a project in NodeJS, I am determined to use Python in its place everytime possible.