1

I have a case that needs to manipulate a large stream of JSON and inject it into Apache HBase. Our system works on Node.js with Mongo then, since we need to enhance performance, so HBase is choosen to handle the big data things.

To enhance my system scalability, I prefer using the Actor Model by Akka for messaging instead of using any other messaging system. It's because Actor Model that Akka provides gives me any advantages about fail safe, Actor management, and other features that's very helpful to make my job easy. But it still in JVM layer that directly injecting and consuming data from HBase.

I want my Node.js apps also works under Akka system maybe using node-java. Is it good practice? If it's not, is there any solution that Node.js can communicate with Akka?

ps. my question here is about how to work with Akka and Node.js, not arguing about "why choose to use Node.js when JVM has really fast JSON manipulating library", because our system are already benchmarked and Node.js was the chosen one to handle JSON manipulation. Also it's already at production stage, so migration totally from Node.js to full Scala is not our priority today.

  • What specific kind of JSON manipulation are you missing on the JVM? – Giorgio Jul 4 '15 at 6:26
  • It's about flexibility, my JSON are streams from any social media and other online sources. It has no fix attributes/ field names, so I must loop trough any existing fields also check each of it's childs if it's an array or JSON object. That JSON from different source is converted into other standard JSON that my other process do into injecting it to MongoDB and HBase later. It's very slow compared to Node.js when loop trough large number of JSON when I use Scala. – Andi N. Dirgantara Jul 4 '15 at 6:30
1

I don't buy the argument that the JVM is not as good to manipulate JSON than Node. The JVM is filled with a lot of really fast JSON parsers, Jackson being one of them. As a bonus it handles streaming json as well. Look at the project before making an opinion. Your problem is fairly common, you want to connect to a firehose of social data, via an actor, pass the entity to another actor to validate or modify to your desired kind of Json, then pass it to another actor to persist to your data store. You might use some queues (LinkedBlockingQueues, etc) in the middle if needed. In the validating and parsing actor, just Jackson or the Json4s project (https://github.com/json4s/json4s) to enrich or modify your JSON. The rest is fairly standard.

  • Ok, I edited my question, I won't arguing about JSON manipulating, my question is about how to connect to Akka from Node.js. Since my system actually already on production stage at some company and it's built with Node.js. – Andi N. Dirgantara Jul 5 '15 at 6:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.