Taking a Live Look: How to Observe Your Node.js Application Logs in Real-Time

Introduction

In this article, I’ll be demonstrating how you can add a “live” console web tail to your Node.js application hosted on BlueMix in a few easy steps without cluttering up your application code. We’ll be achieving this by overloading the Console object, something you find in almost every javascript program. This is a handy feature for real-time debugging as well as monitoring your running applications. It also serves as a prototype for modifying console statements or adding entirely new console types. I’ll be assuming you have at least have a basic familiarity with Node.js. The instructions vary slightly depending on if you’d like to use an Express server or an HTTP server, so I have provided a set of steps and an example for both. The inspiration for this project/hack was a blog post about overloading the console functions while preserving their functionality found here by Udi Talias.

Before we start, here are the links for the finished Express example and the finished HTTP example.

Steps for Express

We’ll start by taking a look at a sample Express application:

Not much more than a basic Hello World Express application, that will create some console.log chatter for us to observe in our web tail. For reference, this is the starting file structure of our project:

Now we can jump into installing our live console with the following steps.

• Modify your package.json file in your project, and add the following dependencies if you aren’t already using them: socket.io and ansi-to-html. For example:

  "dependencies":<br> {<br>     "express": "3.4.7",<br>     "socket.io": "*",<br>     "ansi-to-html": "0.1.1"<br> }<br>

  Scroll to view full table

• Create two folders in your project folder, named lib and public. Drop console.js into lib and drop tail.html in public. Your file structure should now look something like this (feel free to ignore my stylesheets folder, it isn’t required) :

  C:\Users\IBM_ADMIN\console tailing&gt;tree /F<br> Folder PATH listing<br> Volume serial number is 1E2C-67D3<br> C:.<br> │   app.js<br> │   manifest.yml<br> │   package.json<br> │<br> ├───lib<br> │       console.js<br> │<br> └───public<br>     │   tail.html<br>     │<br>     └───stylesheets<br>             index.css<br>

  Scroll to view full table

• Modify lines 13 and 16 in tail.html so that they point to the URL of your BlueMix application, for example:

  &lt;script src="https://.ng.bluemix.net/socket.io/socket.io.js"&gt;<p></p> <p>var socket = io.connect("https://.ng.bluemix.net");<br> </p>

  Scroll to view full table

• The only thing left to do is modify our app.js slightly. What we’ll need to do is

  • add this line so that your Express server will serve static files from your public directory:

    app.use(express.static(__dirname + '/public'));<br>

    Scroll to view full table

  • replace the line of code telling your app to listen on the removed port and host variables with a line requiring console.js and passing your Express server to it as a parameter.

    // app.listen(port, host);<br> var custom_console = require('./lib/console.js')(app);<br>

    Scroll to view full table

  So that when you’re all done, it looks like this:

  'use strict';<br> // dependencies<br> var http        =  require('http')<br>   , express     =  require('express');<p></p> <p>// port and host<br> var host        = process.env.VCAP_APP_HOST || process.env.HOST || 'localhost';<br> var port        = process.env.VCAP_APP_PORT || process.env.PORT || 3000;</p> <p>// create Express server<br> var app = express();</p> <p>// set up middleware<br> app.use(express.bodyParser());<br> // line for serving static files from public directory<br> app.use(express.static(__dirname + '/public'));<br> app.use(express.logger('dev'));</p> <p>// endpoint that creates logs to demonstrate web tail<br> app.get('/', function(req, res){<br>     setInterval(function() {<br>         console.log("you would be seeing the logs of your app");<br>     }, 3000);<br>     setInterval(function() {<br>         console.log("right here.");<br>     }, 1500);<br>     setInterval(function() {<br>         console.log("as it runs");<br>     }, 1500);<br>     setInterval(function() {<br>         console.log("isn't that exciting?");<br>     }, 5000);<br>     res.end("Hello world.  I'm demonstrating the functionality of /dash.html by logging test messages on an interval.");<br> });</p> <p>// app.listen(port, host);<br> // require to connect console.js by passing the express server<br> // Anonymous Function that prefixes timestamps to console.log<br> var custom_console = require('./lib/console.js')(app);<br> </p>

  Scroll to view full table

And we’re done! Jump down to the Result’s section to test drive it.

Steps for HTTP

We’ll start by taking a look at a sample Node.js application using an HTTP server:

Not much more than a basic Hello World application, that will create some console.log chatter for us to observe in our web tail. For reference, this is the starting file structure of our project:

Now we can jump into installing our live console with the following steps.

• Modify your package.json file in your project, and add the following dependencies if you aren’t already using them: socket.io and ansi-to-html. For example:

  "dependencies":<br> {<br>     "express": "3.4.7",<br>     "socket.io": "*",<br>     "ansi-to-html": "0.1.1"<br> }<br>

  Scroll to view full table
• Create two folders in your project folder, named lib and public. Drop console.js into lib and drop tail.html in public. Your file structure should now look something like this (feel free to ignore my stylesheets folder, it isn’t required) :

• Modify lines 13 and 16 in tail.html so that they point to the URL of your BlueMix application, for example:

• The only thing left to do is modify our app.js slightly. What we’ll need to do is

  • add these lines to your callback function passed when creating your HTTP server, in order to conditionally determine if you are navigating to tail.html or not. We do this so that tail.html can be served from our public directory:

    // detect if we are navigating to the tail or not<br> if (req.url == "/tail.html") {<br>     fs.readFile(__dirname + '/public/tail.html',<br>     function (err, data) {<br>     if (err) {<br>       res.writeHead(500);<br>       return res.end('Error loading tail.html');<br>     }<br>     res.writeHead(200);<br>     res.end(data);<br>     });<br> }<br>

    Scroll to view full table

  • replace the line of code telling your app to listen on the removed port and host variables with a line requiring console.js and pass your HTTP server to it as a parameter.

    // app.listen(port, host);<br> var custom_console = require('./lib/console.js')(app);<br>

    Scroll to view full table

  So that when you’re all done, it looks like this:

  'use strict';<br> // dependencies<br> var app = require('http').createServer(handler)<br>   , fs = require('fs');<p></p> <p>// port and host<br> var host        = process.env.VCAP_APP_HOST || process.env.HOST || 'localhost';<br> var port        = process.env.VCAP_APP_PORT || process.env.PORT || 3000;</p> <p>function handler (req, res) {<br>     // detect if we are navigating to the tail or not<br>     if (req.url == "/tail.html") {<br>         fs.readFile(__dirname + '/public/tail.html',<br>         function (err, data) {<br>         if (err) {<br>           res.writeHead(500);<br>           return res.end('Error loading tail.html');<br>         }<br>         res.writeHead(200);<br>         res.end(data);<br>         });<br>     } else {<br>         // if not tail then generic endpoint<br>         res.writeHead(200);<br>         triggerConsole();<br>         res.end("Hello world!  I am demonstrating the live web console tail.  Navigate to /tail.html and check it out.");<br>     }<br> }</p> <p>// dummy function used to recreate standard application log "chatter"<br> function triggerConsole() {<br>     setInterval(function() {<br>         console.log("you would be seing the logs of your app");<br>     }, 3000);<br>     setInterval(function() {<br>         console.log("right here.");<br>     }, 1500);<br>     setInterval(function() {<br>         console.log("as it runs");<br>     }, 1500);<br>     setInterval(function() {<br>         console.log("isn't that exciting?");<br>     }, 5000);<br> }</p> <p>// app.listen(port, host);<br> // line requiring the external file lib/console.js passing it the expected parameters<br> var timestamp_console = require('./lib/console.js')(app);<br> </p>

  Scroll to view full table

And we’re done! Jump down to the Result’s section to test drive it.

Checking Out the Result

It’s time to see the fruits of our labor. Push your example of choice to BlueMix. Now navigate to your application’s route and append /tail.html to it. You should be greeted by the web tail:

You can confirm everything is working properly by opening your application’s route in another tab and hitting that endpoint to trigger some logging functions. You should see your tail.html populate itself with logs as they are called by your application. You can now debug your app in real-time or monitor it’s usage on the fly.