Advanced Node.js Hacks: Boost Your Development Skillset

··

27 min read

The Node.js binary
- Using command-line flags to alter Node.js behavior
- The process object and its role in Node.js applications
Debugging and diagnostics
Key JavaScript concepts
Packages and dependencies
Node's module systems
Node's event system
Handling errors
Using buffers
Working with streams
Interacting with the file system
Process and the Operating System
Creating child processes
Writing unit tests
- Assertion libraries
- Writing tests
Conclusion

As I gear up for the Node.js Application Developer certification (JSNAD), I've turned my study notes into this blog post to help supercharge your Node.js skills.

Think of it as a friendly guide, written for fellow developers who are on the same path or anyone else interested in diving into Node.js.

From the basic building blocks like the Node binary to more advanced topics like stream handling and child processes, it's a series of valuable insights and practical tips aimed at giving your Node.js development skills a significant boost.

My hope is that these notes become a resource you can rely on to enhance your coding practice, providing you with impactful takeaways.

The Node.js binary

Binaries and executable files are the final, runnable versions of software programs, translated from source code into a language that the computer's processor can understand and act upon.
Think of it as a ready-to-eat meal. Think of the source code as a recipe that contains instructions on how to prepare a dish. However, you cannot eat a recipe; it needs to be followed and executed by cooking the dish.

The Node.js binary, usually invoked with node in the command line, is what runs your Node.js scripts. It's compiled from C++ code and bridges the gap between your JavaScript code and the underlying system operations.

To run a simple JavaScript file named app.js, you would use the following command in the terminal:

node app.js

Using command-line flags to alter Node.js behavior

Node.js can be customized at runtime using various command-line flags. These flags can change the behavior of the Node.js binary and affect how your scripts are executed.

To inspect the memory usage of a Node.js application, you might use the --inspect flag. This command will run your app.js script and enable the inspector agent which provides a debugger interface for Node.js applications.

node --inspect app.js

Another practical flag is --use_strict which enforces JavaScript's strict mode globally:

node --use_strict app.js

The `process` object and its role in Node.js applications

The process object is a global that provides information about, and control over, the current Node.js process. It's an instance of EventEmitter and is central to Node.js scripting.

You can use process to access environment variables, which is common for configuring applications based on the development environment.

// Outputs the environment Node.js is running in,
// like 'development' or 'production'
console.log(process.env.NODE_ENV);

Another example is gracefully shutting down your application when a certain signal is received:

process.on('SIGTERM', () => {
  console.log('Process terminated');
  process.exit(0);
});

Here's an example that prints out arguments passed from the command line:

// Save this as args-demo.js
// Run in terminal with:
// node args-demo.js one two=three four
process.argv.forEach((val, index) => {
  console.log(`${index}: ${val}`);
});

To which the output would be:

0: path/to/node/binary
1: path/to/your/args-demo.js
2: one
3: two=three
4: four

In Node.js, the process object is also used to handle uncaught exceptions and cleanup before the process exits, among other things, making it a powerful interface for process-related operations.

Debugging and diagnostics

Debugging is the process of finding and resolving defects or problems within our code.
Imagine your car isn't starting. To find the root cause, you would go through a systematic process of elimination: checking the battery, the spark plugs, the fuel supply, and so on.

It's often helpful to start with simple console logging to understand the flow and state of your application, then move on to the debugger for more complex issues, and use performance tools to optimize and ensure efficiency.

Console methods for basic debugging

Node.js provides a console module which is similar to the JavaScript console object provided by web browsers. It includes methods for logging, asserting, and tracking execution times.

Logging

console.log('Hello, World!'); // Basic logging
console.error('Error: Something went wrong'); // Logging errors

Assertions

console.assert(1 === 2, 'This will fail and print an assertion error');

Tracking execution times

console.time('loop'); // Starts a timer with a name "loop"
for (let i = 0; i < 1000000; i++) { /* Loop operation */ }
console.timeEnd('loop'); // Ends the timer "loop" and logs the elapsed time

Node.js core debugger

Node.js includes an out-of-process debugging utility accessible via a web inspector or command-line interface. You can start Node.js in debug mode or attach a debugger to an already running process.

Starting Node.js in debug mode

This command will start the Node.js process in debug mode and pause execution until a debugger is attached.

node --inspect-brk app.js

Using the built-in debugger CLI

Here you enter a REPL where you can set breakpoints, step through the code, and inspect variables.

node inspect app.js

Performance analysis using Node.js tools

Node.js offers built-in tools for performance analysis, such as the --inspect flag, which can be used in conjunction with Chrome Developer Tools, and performance hooks available via the perf_hooks module.

Profiling with the inspector

This will profile your Node.js application and you can then open Chrome Developer Tools to view the CPU profile.

node --inspect --prof app.js

Measuring performance with `perf_hooks`

const { performance } = require('perf_hooks');

let startTime = performance.now();

// Some synchronous or asynchronous operation
for (let i = 0; i < 1000000; i++) { /* ... */ }

let endTime = performance.now();

console.log(`Operation took ${endTime - startTime} milliseconds`);

Key JavaScript concepts

Scope, closures, and the event loop

Scope

In JavaScript, scope refers to the current context of code, which determines the accessibility of variables to JavaScript entities like functions, blocks, and expressions.

let globalVar = 'global';

function checkScope() {
  let localVar = 'local';
  console.log(globalVar); // Outputs: global
  console.log(localVar);  // Outputs: local
}

checkScope();
console.log(localVar); // Uncaught ReferenceError: localVar is not defined

Closures

A closure is a function that remembers the variables from the place where it is defined, regardless of where it is executed later.

function createCounter() {
  let count = 0;
  return {
    increment: function() {
      count++;
    },
    currentValue: function() {
      return count;
    }
  };
}

const counter = createCounter();
counter.increment();
console.log(counter.currentValue()); // Outputs: 1

Event loop

The event loop is a mechanism that allows Node.js to perform non-blocking I/O operations, despite JavaScript being single-threaded, by offloading operations to the system kernel whenever possible.

console.log('First');

setTimeout(() => {
  console.log('Second');
}, 0);

console.log('Third');
// Outputs: First, Third, Second

Async programming and promises

Asynchronous programming in JavaScript is performed using callbacks, promises, and async/await.

Promises

A Promise is an object representing the eventual completion or failure of an asynchronous operation.

let promiseToCleanTheRoom = new Promise(function(resolve, reject) {
  // cleaning the room
  let isClean = true;
  if (isClean) resolve('Clean');
  else reject('Not Clean');
});

promiseToCleanTheRoom
  .then(function(fromResolve) {
    console.log('the room is ' + fromResolve);
  })
  .catch(function(fromReject) {
    console.log('the room is ' + fromReject);
  });

ES6+ features heavily used in Node.js

ES6 (ECMAScript 2015) and beyond have introduced many features that have become heavily used in Node.js. These are the fundamental building blocks for understanding more advanced Node.js functionality.

let and const

let a = 'Hello'; // Block-scoped variable
const b = 'World'; // Constant reference

Arrow functions

const add = (a, b) => a + b;

Template literals

const name = 'World';
console.log(`Hello, ${name}!`); // Outputs: Hello, World!

Destructuring

const person = { name: 'John', age: 30 };
const { name, age } = person;

Default parameters

function greet(name = 'World') {
  console.log(`Hello, ${name}!`);
}
greet(); // Outputs: Hello, World!

Async/await for promises

async function fetchUsers() {
  let users = await getUsersFromDatabase();
  console.log(users);
}

Packages and dependencies

Dependencies are components or pieces of code that a software needs. These could be libraries, frameworks, or other software modules that the project relies on.
It's like ingredients in a recipe. Just as a recipe for a cake requires certain ingredients like flour, sugar, eggs, and butter, a software project might need specific libraries or tools to work.

Packages are the way to create libraries that others and yourself can reuse while building applications.

The `package.json` file and its main properties

The package.json file serves as the central manifest for your application. It specifies the metadata for your project and lists the dependencies required.

This is an example package.json file:

{
  "name": "my-nodejs-app",
  "version": "1.0.0",
  "description": "A sample Node.js app",
  "main": "app.js",
  "scripts": {
    "start": "node app.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Jane Doe",
  "license": "ISC",
  "dependencies": {
    "express": "^4.17.1",
    "mongoose": "^5.9.10"
  },
  "devDependencies": {
    "nodemon": "^2.0.3",
    "jest": "^26.0.1"
  }
}

And these are the main properties, explained:

name: The application’s name.
version: Current version of the application.
description: A brief description of the application.
main: The entry point of the application.
scripts: Scripts to automate repetitive tasks like starting, testing, and building the app.
author: Information about the author.
license: The license type.
dependencies: Packages required for production.
devDependencies: Packages required for development and testing.

Managing packages with NPM

NPM stands for Node Package Manager and it is the default package manager for Node.js. It handles the installation and management of packages from the npm registry.

Installing a package

This command will install the lodash package and add it to the dependencies in your package.json. View the documentation for this command.

npm install lodash
npm i lodash

Installing a package as a dev dependency

Add either -D or --save-dev flag to the install command to add the package to the devDependencies instead, meaning this package will not be bundled for production deployments. View the documentation for this command.

npm install -D lodash
npm i -D lodash

Updating a package

This updates lodash to the latest version according to the version rules in your package.json. View the documentation for this command.

npm update lodash

Uninstalling a package

This removes lodash from your node_modules and package.json. View the documentation for this command.

npm uninstall lodash

Semantic versioning and dependency management

Semantic versioning (aka. semver) is a set of conventions for versioning software in the format of MAJOR.MINOR.PATCH. npm uses this system to manage package versions and handle version compatibility.

Examples:

^1.0.0: Allows updates that do not modify the left-most non-zero digit (1.x.x). In practice, it will allow patch and minor updates, but not major updates, meaning that you'll get bug fixes and new features but not breaking changes.
~1.0.0: Allows patch-level changes that do not modify the left-most non-zero digit (1.0.x). In practice, it will only allow patch updates, blocking new features and breaking chages, meaning that you'll get bug fixes but not new features or breaking changes.
1.0.0: Pins the version to exactly 1.0.0.

Read the documentation on semver to learn more.

The `package-lock.json` file

Then we also have the package-lock.json file. This file is a manifest created by package managers like npm or Yarn, which precisely defines the versions of each dependency and sub-dependency installed in a Node.js project.

For npm, this file is named package-lock.json, and for Yarn, it's yarn.lock.

The lock file ensures that the same versions of the dependencies are installed in every environment, thereby avoiding discrepancies between different development environments and production.

This feature is critical for consistency, as it mitigates issues arising from version updates of dependencies and allows developers to have a synchronized dependency tree across different machines, including continuous integration servers.

The lock file also optimizes the installation process by providing a clear installation blueprint, which can speed up package installation in CI/CD pipelines.

Important: you should version this file!

Node's module systems

A module or package system is code that is organized into separate units (modules) or collections (packages), which can be used and reused across different parts of a program or even between different programs. They encapsulate functionality and can be easily imported or included where needed.
A real-world analogy to a module or package system is a library system. In a library, books are organized into sections according to subject matter, genre, or author. When you need information on a specific topic, you go to the corresponding section and borrow the books (modules/packages) that contain the information you need.

Node.js uses the CommonJS module system for encapsulating and organizing code into reusable components.

In CommonJS, each JavaScript file is treated as a separate module. Each module can export objects, functions, or variables to be used in other files.

Example of a CommonJS module (math.js)

// Define a couple of functions and a constant
const PI = 3.14159;

function add(x, y) {
  return x + y;
}

function subtract(x, y) {
  return x - y;
}

// Export them
module.exports = {
  add,
  subtract,
  PI
};

And how to use that module:

// Import the entire math module
const math = require('./math.js');
console.log(math.add(2, 3));       // Outputs: 5
console.log(math.subtract(5, 2));  // Outputs: 3
console.log(math.PI);              // Outputs: 3.14159

More about the `require` function

The require function is used to load modules. It reads the module file, executes the file within its own scope, and returns the module.exports object.

Example with destructuring

// Import only the needed functions
const { add, subtract } = require('./math.js');
console.log(add(6, 4));            // Outputs: 10
console.log(subtract(10, 5));      // Outputs: 5

Example with dynamic require

// moduleName could be 'math.js' or any other module name
const moduleName = 'math.js';
const math = require(`./${moduleName}`);

Module caching and circular dependencies

Modules are cached after the first load. This means that every subsequent require call for a previously loaded module will return exactly the same exported object, not a fresh copy.

// If math.js is required elsewhere, it will not be re-executed
const mathFirstLoad = require('./math.js');
const mathSecondLoad = require('./math.js');
console.log(mathFirstLoad === mathSecondLoad); // Outputs: true

Handling circular dependencies

When two or more modules require each other, this is known as a circular dependency. Node.js handles this by loading a partial export of the module if it's still being executed.

In the following example, if a.js requires b.js and b.js requires a.js, Node.js will not get into an infinite loop. Instead, it handles the circular reference by providing the current state of the exported object.

// a.js
const b = require('./b.js');
exports.loaded = true;

// b.js
const a = require('./a.js');
console.log(a.loaded); // Outputs: undefined if a.js hasn't finished executing
exports.loaded = true;

Asynchronous control flow

Node.js is designed to handle asynchronous operations, allowing you to execute tasks like reading files, querying a database, or requesting resources over the network without blocking the main thread.

Callbacks and their drawbacks

Callbacks are functions that are passed as arguments to other functions to be executed after the completion of an asynchronous operation.

const fs = require('fs');

fs.readFile('example.txt', 'utf8', (err, data) => {
  if (err) {
    console.error("Error reading file:", err);
    return;
  }
  console.log(data);
});

The drawbacks of callbacks are:

Callback hell: Nesting too many callbacks can lead to code that is hard to read and maintain, often referred to as "callback hell" or "the pyramid of doom."
Error handling: Errors must be checked and handled manually at every level of the callback chain.
Control flow: Managing the sequence and concurrency of multiple asynchronous operations can become complex.

Promises and async/await for managing asynchronous code

Promises are objects that represent the eventual completion or failure of an asynchronous operation. They can be chained and used to avoid callback hell.

const fsPromises = require('fs').promises;
fsPromises.readFile('example.txt', 'utf8')
  .then(data => console.log(data))
  .catch(err => console.error("Error reading file:", err));

But we also have async/await, which is syntactic sugar built on top of promises, making asynchronous code look and behave like synchronous code.

const fsPromises = require('fs').promises;

async function readAndDisplayFile() {
  try {
    const data = await fsPromises.readFile('example.txt', 'utf8');
    console.log(data);
  } catch (err) {
    console.error("Error reading file:", err);
  }
}

readAndDisplayFile();

Control flow abstractions with `async.series` and `async.waterfall`

Libraries like async provide functions that abstract away the complexities of asynchronous control flow.

The async.serieshelper runs an array of functions in sequence, each one running once the previous function has been completed.

const async = require('async');

async.series([
  function(callback) {
    // Do some asynchronous operation, then...
    callback(null, 'one');
  },
  function(callback) {
    // Do another asynchronous operation, then...
    callback(null, 'two');
  }
],
// Optional callback
function(err, results) {
  // Results is now equal to ['one', 'two']
});

The async.waterfall helper in turn runs an array of functions in sequence, where each function passes its results to the next.

const async = require('async');

async.waterfall([
  function(callback) {
    callback(null, 'one', 'two');
  },
  function(arg1, arg2, callback) {
    // arg1 is 'one' and arg2 is 'two'
    callback(null, 'three');
  },
  function(arg1, callback) {
    // arg1 is 'three'
    callback(null, 'done');
  }
], function (err, result) {
  // result is 'done'
});

Node's event system

An event system is a way of managing actions that occur in response to various actions or changes in state. It's a pattern that enables decoupling of components, so that when a certain event occurs, the system can react by triggering specific behavior without the different parts of the system needing to know about each other.
Consider when guests at a wedding raise their glasses for a toast. This action triggers several responses: the band starts playing a song, the photographer takes pictures, and so on.

Node.js's event-driven architecture is powered by the EventEmitter class, which is part of the events module.

The `EventEmitter` class

The EventEmitter class is used to handle events and listeners asynchronously. It is the cornerstone of many built-in Node.js modules like http, stream, and others.

const EventEmitter = require('events');
const myEmitter = new EventEmitter();

// Event listener
myEmitter.on('event', () => {
  console.log('an event occurred!');
});

// Trigger the 'event'
myEmitter.emit('event');

Creating custom event emitters

You can create your own classes that inherit from EventEmitter to emit and listen for events in your applications.

class MyEmitter extends EventEmitter {}

const myEmitter = new MyEmitter();

myEmitter.on('event', function(a, b) {
  console.log(a, b, this, this === myEmitter); // Prints the arguments and checks the context
});

myEmitter.emit('event', 'arg1', 'arg2');

Error handling and memory leaks in Event Emitters

Proper error handling and memory management are essential when working with EventEmitter to prevent crashes and leaks.

myEmitter.on('error', (err) => {
  console.error('An error occurred:', err);
});

// Emitting an 'error' event
myEmitter.emit('error', new Error('whoops!'));

Memory leak example

By default, event listeners will be added indefinitely. To avoid memory leaks, you can use setMaxListeners and remove listeners when they are no longer needed.

myEmitter.setMaxListeners(100); // Set the limit of listeners to 100

// Remove specific listener
const callback = () => console.log('Event occurred!');
myEmitter.on('event', callback);
// ...
myEmitter.removeListener('event', callback);

// Or remove all listeners for an event
myEmitter.removeAllListeners('event');

Avoiding memory leaks

Use once to add a one-time listener, which is automatically removed after it is invoked.

myEmitter.once('event', () => {
  console.log('This will only fire once');
});

myEmitter.emit('event');
myEmitter.emit('event'); // The listener will not be called this time

Handling errors

Let's explore the best practices for handling errors, including error-first callbacks, try/catch with asynchronous code, and global error handling.

Error-first callbacks

The error-first callback pattern is a convention in Node.js where the first argument to any callback function is an error object. This pattern allows for easier error checking and handling.

const fs = require('fs');

fs.readFile('non-existent-file.txt', 'utf8', (err, data) => {
  if (err) {
    return console.error('Error reading file:', err);
  }
  console.log(data);
});

Try/catch with asynchronous code

The try/catch block does not work with asynchronous callbacks. You need promises and async/await to use it.

const fsPromises = require('fs').promises;

async function readConfigFile() {
  try {
    const data = await fsPromises.readFile('/path/to/config.json', 'utf8');
    const config = JSON.parse(data);
    console.log(config);
  } catch (err) {
    console.error('Error reading or parsing file:', err);
  }
}

readConfigFile();

Global error handling

Global error handling allows you to catch any errors that were not anticipated and handled in local scopes.

process.on('uncaughtException', (err) => {
  console.error('There was an uncaught error:', err);
  // Perform cleanup or graceful shutdown if necessary
  process.exit(1); // Exit with a 'failure' code
});

// This will trigger the 'uncaughtException' event
setTimeout(() => {
  throw new Error('Oops!');
}, 1000);

Warning: using uncaughtException to resume normal operation is not recommended as it might leave your application in an unknown state. It should be used for cleanup tasks and logging before a graceful shutdown.

Best practice

It's better to handle errors locally where they occur and use global handlers as a last resort. Also, consider using newer features like async_hooks for context propagation or error boundaries in frameworks like Express.

Using buffers

A buffer is a region of memory used to temporarily hold data while it is being moved from one place to another. It's especially useful in situations where data is being produced at a different rate than it's being consumed, or when it needs to be received in batches.
Consider a waiting area in a restaurant. Customers (data) arrive at the restaurant and wait in the waiting area (buffer) until a table (the program needing the data) becomes free.

Buffers are used to work with binary data. They allow manipulation of raw data similar to arrays of integers but correspond to raw memory allocations outside the V8 JavaScript engine.

Buffers play a critical role in handling binary data and interact seamlessly with streams, which are fundamental for reading from and writing to files, handling network communications, or any other operation that involves handling IO in a memory-efficient way.

Using buffers effectively ensure that Node.js applications can handle file uploads, image processing, and any other binary data requirements robustly.

Working with the `Buffer` class to handle binary data

Buffers are instances of the Buffer class, which is a global, meaning it's available without importing it.

// Allocates a new Buffer containing the string "Node.js"
const buf = Buffer.from('Node.js', 'utf-8');

// Logging the buffer will show the raw memory allocated for this string
console.log(buf);  // <Buffer 4e 6f 64 65 2e 6a 73>

Encoding and decoding buffers

When you create a Buffer, you can specify an encoding, like UTF-8, and when you convert a Buffer to a string, you can specify an encoding to decode the bytes accordingly.

// Create a Buffer with a string using UTF-8 encoding (default)
const buf = Buffer.from('Buffer creation example', 'utf-8');

// Decode buffer to a string with the same encoding
const str = buf.toString('utf-8');

console.log(str);  // Outputs: Buffer creation example

Buffer and stream interaction

Buffers can be used with streams, which is especially useful for handling or transforming data streams.

Example of interacting with a readable stream

const fs = require('fs');

// Create a readable stream
const readableStream = fs.createReadStream('file.txt');
const dataChunks = [];

// Handle the 'data' event by accumulating chunks of data
readableStream.on('data', (chunk) => {
  dataChunks.push(chunk);
});

// Handle the 'end' event to combine the chunks
readableStream.on('end', () => {
  const completeData = Buffer.concat(dataChunks);
  console.log(completeData.toString('utf-8'));
});

// Handle errors
readableStream.on('error', (err) => {
  console.error('Error:', err);
});

Example of a writable stream with Buffer

const fs = require('fs');

// Create a writable stream
const writableStream = fs.createWriteStream('output.txt');

// Create a buffer with some data
const buf = Buffer.from('This will be written to the file', 'utf-8');

// Write buffer to the stream
writableStream.write(buf);

// End the stream
writableStream.end();

Working with streams

A stream is a sequence of data elements made available over time. It's a way of handling data that allows for the start of processing before the entire data is available. It's particularly useful for handling large data sets or data that's being transmitted live.
Take a conveyor belt at a factory. Items (data elements) are placed on the conveyor belt and move towards processing (the program processing the data).

Streams are collections of data that might not be available all at once and don't have to fit in memory. This makes streams a powerful way of handling data that is fetched or delivered in chunks, like files or network requests.

By using streams, you can process large amounts of data efficiently and without consuming a lot of memory, which is perfect for applications like file processors, data transformers, and network handlers.

https://phillcode.io/nodejs-stream

Readable and writable stream interfaces

Readable streams allow you to read data from a source in chunks, and writable streams allow you to write data to a destination in chunks.

Example of a readable stream

const fs = require('fs');

const readableStream = fs.createReadStream('input.txt', {
  encoding: 'utf8',
  highWaterMark: 16 * 1024 // 16KB chunk size
});

readableStream.on('data', function(chunk) {
  console.log('New chunk received:');
  console.log(chunk);
});

Example of a writable stream

const fs = require('fs');

const writableStream = fs.createWriteStream('output.txt');

writableStream.write('Hello\n');
writableStream.write('World\n');
writableStream.end('Ending the write\n');

writableStream.on('finish', () => {
  console.log('Write completed.');
});

Piping streams

Piping is a method to connect the output of a readable stream to the input of a writable stream.

const fs = require('fs');

const readableStream = fs.createReadStream('input.txt');
const writableStream = fs.createWriteStream('output.txt');

// Pipe the read and write operations
readableStream.pipe(writableStream);

writableStream.on('finish', () => {
  console.log('Piping completed.');
});

Transform streams and duplex streams

Transform streams are a type of duplex stream that can be used to modify data as it is written and read. Duplex streams are streams that are both readable and writable.

Example of a transform stream

In this example, any input from stdin is converted to uppercase and then sent to stdout.

const { Transform } = require('stream');

const upperCaseTr = new Transform({
  transform(chunk, encoding, callback) {
    // Convert chunk to uppercase
    this.push(chunk.toString().toUpperCase());
    callback();
  }
});

process.stdin.pipe(upperCaseTr).pipe(process.stdout);

Example of a duplex stream

In this duplex stream example, it reads input from stdin, logs it to the console, and also writes a sequence of letters to stdout.

const { Duplex } = require('stream');

const inoutStream = new Duplex({
  write(chunk, encoding, callback) {
    console.log(chunk.toString());
    callback();
  },
  read(size) {
    if (this.currentCharCode > 90) {
      this.push(null);
      return;
    }
    this.push(String.fromCharCode(this.currentCharCode++));
  }
});

inoutStream.currentCharCode = 65; // ASCII code for 'A'
process.stdin.pipe(inoutStream).pipe(process.stdout);

Interacting with the file system

Node.js provides the fs module, which allows interaction with the file system in various ways, from reading and writing files to accessing their metadata.

Reading from and writing to files

The fs module contains methods for both synchronous and asynchronous file operations.

Asynchronous file read example

const fs = require('fs');

fs.readFile('input.txt', 'utf8', (err, data) => {
  if (err) {
    return console.error(err);
  }
  console.log(data);
});

Synchronous file write example

const fs = require('fs');

try {
  fs.writeFileSync('output.txt', 'Here is some text', 'utf8');
  console.log('File written successfully');
} catch (err) {
  console.error(err);
}

Working with file metadata

Node.js allows you to access file metadata, such as file size, creation and modification dates, and permissions.

const fs = require('fs');

fs.stat('input.txt', (err, stats) => {
  if (err) {
    return console.error(err);
  }
  console.log(stats);
  console.log(`File Size: ${stats.size} bytes`);
  console.log(`Created on: ${stats.birthtime}`);
  console.log(`Last Modified: ${stats.mtime}`);
});

File streaming for efficient data handling

Streams are an efficient way to handle data, especially if you're dealing with large files, because they don't require you to load the entire file into memory.

Example of reading a file with a stream

const fs = require('fs');
const readStream = fs.createReadStream('largefile.txt', 'utf8');

readStream.on('data', (chunk) => {
  console.log('Reading a chunk of data:');
  console.log(chunk);
});

readStream.on('end', () => {
  console.log('Finished reading file.');
});

Example of writing to a file with a stream

const fs = require('fs');
const data = 'Some data to write to the file stream\n';
const writeStream = fs.createWriteStream('output.txt', 'utf8');

writeStream.write(data, 'utf8');
writeStream.end();

writeStream.on('finish', () => {
  console.log('Finished writing to file.');
});

writeStream.on('error', (err) => {
  console.log(err.stack);
});

Example of piping between readable and writable streams

const fs = require('fs');

// This will read from 'input.txt' and write directly to 'output.txt'
fs.createReadStream('input.txt')
  .pipe(fs.createWriteStream('output.txt'));

Process and the Operating System

By interacting with the operating system (OS), a software can communicate directly with the system's hardware and resources, often bypassing abstractions and simplifications provided by higher-level functions.
A good example might be the interaction between a factory worker and the factory machinery. Workers need to be aware of the detailed workings of the factory's systems, like the power supply, raw materials and other things to operate the machines efficiently.

In Node.js, the process object is a global that provides information about, and control over, the current Node.js process. Moreover, Node.js provides modules like os and child_process for interacting with the operating system.

Node.js is designed to facilitate the development of fast and scalable network applications. By leveraging the process object and other OS-related modules, we can create applications that interact intelligently with the operating system and the environment in which they run.

This includes handling environment configuration, accessing system-level information, and managing the process lifecycle.

The global `process` object

The process object is an instance of EventEmitter and provides access to the running Node.js process.

// Print the current version of Node.js
console.log(`This process is running Node.js version ${process.version}`);

// Access the current working directory
console.log(`Current directory: ${process.cwd()}`);

// Get the process id
console.log(`This process has pid ${process.pid}`);

// Exit the process
process.exit(0);

Interacting with the OS

Node.js's os module provides a way to interact with the underlying operating system.

const os = require('os');

// Get OS platform (e.g., 'darwin', 'win32', 'linux')
console.log(`Operating system: ${os.platform()}`);

// Get total system memory
console.log(`Total memory: ${os.totalmem()} bytes`);

// Get system uptime
console.log(`Uptime: ${os.uptime()} seconds`);

Environment variables and process states

Environment variables are used to store configuration settings and system behavior outside of the application code.

Example of using environment variables

// Access an environment variable
console.log(`The user's home directory is ${process.env.HOME}`);

// Set an environment variable
process.env.NODE_ENV = 'production';

// Conditional behavior based on environment variables
if (process.env.NODE_ENV === 'development') {
  console.log('Logging additional debug information...');
}

Example of process states

// Listening for the process to exit
process.on('exit', (code) => {
  console.log(`About to exit with code: ${code}`);
});

// Handling uncaught exceptions
process.on('uncaughtException', (err) => {
  console.error(`Uncaught exception: ${err}`);
  process.exit(1);
});

Creating child processes

Child processes have their own separate memory space and resources. They are isolated from each other, which means they don't share the same memory or resources directly but can still communicate through specific inter-process communication mechanisms.
It's like setting up separate food stalls at a festival, each with its own supplies and resources. If one stall runs out of supplies or has any kind of problem, it doesn't directly affect the others.

Node.js can create child processes to perform parallel tasks, utilize additional system resources, or manage separate tasks outside of the main Node.js event loop using the child_process module.

The `spawn` method

The spawn method is used to launch a new process with a given command. It streams the data returned from the child process, which makes it ideal for large amounts of data.

const { spawn } = require('child_process');
const child = spawn('ls', ['-lh', '/usr']); // Lists directory contents

child.stdout.on('data', (data) => {
  console.log(`stdout: ${data}`);
});

child.stderr.on('data', (data) => {
  console.error(`stderr: ${data}`);
});

child.on('close', (code) => {
  console.log(`child process exited with code ${code}`);
});

The `exec` method

This one is used to execute a command in a shell and buffer the data returned. It is better suited for small amounts of data, as it buffers the output in memory.

const { exec } = require('child_process');

exec('cat *.js missing_file | wc -l', (error, stdout, stderr) => {
  if (error) {
    console.error(`exec error: ${error}`);
    return;
  }
  console.log(`stdout: ${stdout}`);
  console.error(`stderr: ${stderr}`);
});

IPC (inter-process communication)

To enable IPC between the parent and child processes, you can use spawn with the { stdio: 'ipc' } option.

const { spawn } = require('child_process');

const child = spawn('node', ['child.js'], {
  stdio: [0, 1, 2, 'ipc']
});

child.on('message', (message) => {
  console.log('Message from child:', message);
});

child.send({ hello: 'world' });

And, in child.js, you might have something like:

process.on('message', (message) => {
  console.log('Message from parent:', message);
});

process.send({ foo: 'bar' });

Managing child process resources

It's important to properly manage the resources used by child processes. This includes handling events like close and exit, and killing a child process if necessary. In other words, to manage their lifecycle, and clean up resources after they have finished.

This ensures that the Node.js application maintains high performance and does not leak resources or leave hanging processes.

const { spawn } = require('child_process');
const child = spawn('some-long-running-process', []);

setTimeout(() => {
  child.kill(); // Kills the process
}, 60000); // Assume the process shouldn't take more than a minute

child.on('exit', (code, signal) => {
  if (code) {
    console.log(`Process exited with code ${code}`);
  }
  if (signal) {
    console.log(`Process killed with signal ${signal}`);
  }
});

Writing unit tests

Unit tests are designed to validate that each piece of the code (the "unit") performs as expected in isolation. We write these tests to automatically check the correctness of our code in the face of constant changes. This helps us to quickly identify and fix errors.
This is like quality checks in a manufacturing process. In a factory, each individual component of a product might be tested to ensure it meets certain standards before it's assembled into the final product.

Node.js supports unit testing through various libraries and frameworks, besides offering its own native methods for enabling unit tests.

Assertion libraries

Node.js has a built-in module called assert that provides a simple set of assertion tests.

const assert = require('assert');

// Function to test
function add(a, b) {
  return a + b;
}

// Test case
assert.strictEqual(add(2, 3), 5, '2 + 3 should equal 5');

Writing tests

Tests can be written straightforwardly, asserting the expected outcomes of functions.

const assert = require('assert');

// Function to test
function multiply(a, b) {
  return a * b;
}

// Test case
try {
  assert.strictEqual(multiply(3, 3), 9, '3 * 3 should equal 9');
  console.log('Test passed!');
} catch (err) {
  console.error('Test failed:', err.message);
}

While the assert module is good for simple assertions, for more complex testing scenarios, you might want to use more powerful libraries like Mocha, Chai, Jest, or others that provide more expressive ways of writing tests, mocks, spies, and stubs.

Check out the following post to learn more about setting up and using Jest as a unit testing framework in your Node.js projects!

https://phillcode.io/jest-config

Conclusion

And that's a wrap on our Node.js study session! I hope my notes can help you as much as they've helped me, whether you're also studying for the JSNAD certification or just looking to brush up on your Node.js skills.

We've covered a lot, from the basics right up to the more complex stuff, and I've tried to keep things straightforward and practical.

Remember, the key to getting better at Node.js (or anything, really) is practice, so go get your hands dirty with some code. Good luck with your coding 🤘

Node.js JavaScript Certification best practices