A chart showing recording size of IPCAM using Ajax & Express

The size of IP camera recording can be significantly influenced by various factors, including configuration settings such as encoding method (H264, H265), bit rate, and video size, as well as environmental conditions like lighting, video noise level and the moving speed of recorded objects. To visually represent these variations, I intend to utilize a line chart that displays changes in recording size over time (in seconds). To accomplish this, I employed an Orange Pi running Ubuntu and setup a FTP server. The IP camera is configured to save video to FTP server. The SD card usage is read by using Linux command “df” and the increase in memory used is the recording size of IPCAM. To read and present these data. I use Node.js, and create an Express server along with a static HTML file that incorporates a third-party graph library (chart,js). The following sections provide the details of this mini project, including code.

(A) Setting a Dahua IPCAM to record the video to a ftp server in Ubuntu through wired network. Please refer to operation menu of the model for detailed configuration. Username refers to the user “cctv” in Ubuntu. The password added here must match with that in Ubuntu. Sever IP is the IP address of Ubuntu

(B) Setting a FTP server in Ubuntu
I wouldn’t provide detailed instructions for installing Ubuntu and Node.js as it would be time-consuming. However, you can refer to my other blogs for in-depth installation guides. To install the FTP server on Ubuntu, you can use the following commands in the Ubuntu terminal.

apt update
apt upgrade
apt install vsftpd

Edit the configuration of vsftpd using nano editor.

sudo nano /etc/vsftpd.conf

Add following lines at the end of the configuration file /etc/vsftpd.conf.

write_enable=YES
chroot_local_user=NO
allow_writeable_chroot=YES
user_sub_token=$USER
local_root=/media/sdcard/$USER

Restart the ftp server

systemctl restart vsftpd

Add a new user “cctv” and its password. This account is used by ipcam to record the video.

adduser cctv

Switch to user cctv and add folders

su - cctv
mkdir /home/cctv/ftp
mkdir /home/cctv/ftp/files

(C) Create a static HTML file “sd_usedsize.html” in the public folder. This HTML file includes the necessary code to fetch the used size of the SD card, update the arrays with the retrieved data, and generate a dynamic line chart using Chart.js. The chart will display the recording size per second against time, updating every 3 seconds and showing the data for the past 3 minutes.

<!DOCTYPE html>
<html>
<head>
  <title>IP Camera Recording Size per second</title>
  <script src="js/jquery-3.7.1.min.js"></script>
  <script src="https://cdn.jsdelivr.net/npm/chart.js"></script>
  <style>
    #chart {
      width: 100%;
      height: 400px;
    }
	
    #dataDisplay {
      margin-top: 20px;
      padding: 10px;
      border: 1px solid #ccc;
      font-family: Arial, sans-serif;
    }
  </style>
</head>
<body>
  <h1>IP Camera Recording Size per second</h1>
  <canvas id="chart"></canvas>
  <div id="dataDisplay"></div>

  <script>
    // Array to store timestamps
    let timestamps = [];

    // Array to store footage sizes
    let footageSizes = [];
	
    // Function to fetch used size and update arrays
    function fetchUsedSize() {
      $.ajax({
        url: '/sdusedsize',
        method: 'GET',
        success: function(response) {
          // Extract the numerical value from the response
          const sizeMatch = response.match(/\d+/);
          const size = sizeMatch ? parseInt(sizeMatch[0]) : 0;

          // Get current timestamp
          const timestamp = Date.now();
		  
          // Add timestamp and footage size to arrays
          timestamps.push(timestamp);
          footageSizes.push(size);

          // Remove oldest data if more than 3 minutes of data
          const fiveMinutesAgo = timestamp - 3 * 60 * 1000;
          while (timestamps[0] < fiveMinutesAgo) {
            timestamps.shift();
     		footageSizes.shift();
          }

          // Update the chart
          generateGraph();
		  
	  // Update the data display
          updateDataDisplay();
        },
        error: function(error) {
          $('#usedSizeList').append('<li>Error retrieving SD card usage</li>');
        }
      });
    }

// Function to generate the line chart
function generateGraph() {
  // Calculate the size difference between now and 5 seconds ago
  const sizeDifferences = [];
  for (let i = 1; i < footageSizes.length; i++) {
    const sizeDifference = (footageSizes[i] - footageSizes[i - 1]) / 3;
    sizeDifferences.push(sizeDifference);
  }
  
  // Determine the starting index based on the length of the data
  const startIndex = Math.max(0, sizeDifferences.length - 60); // Show the last 60 data points
 
  // Create an array of labels for specific grid lines
  const labels = [0, '', '', '', '', 15, '', '', '', '', 30, '', '', '', '', 45, '', '', '', '', 60, '', '', '', '', 75, '', '', '', '', 90, '', '', '', '', 105, '', '', '', '', 120, '', '', '', '', 135, '', '', '', '', 150, '', '', '', '', 165, '', '', '', '', 180];
  
    // Prepare data for the chart
  const data = {
    labels: labels,
    datasets: [
      {
        label: 'Recording Size (KBytes)',
        data: sizeDifferences.slice(startIndex),
        borderColor: 'rgba(75, 192, 192, 1)',
        backgroundColor: 'rgba(75, 192, 192, 0.2)',
        tension: 0 //0 is straight line >0 is curved
      }
    ]
  };

  // Check if the chart instance already exists
  if (window.chartInstance) {
    const chart = window.chartInstance;
    chart.data.labels = labels; // Update the labels

    // Update the dataset with the new data
    chart.data.datasets[0].data = sizeDifferences.slice(startIndex);

    // Shift the x-axis grid lines by one position to the left
    chart.options.scales.x.ticks.min = chart.options.scales.x.ticks.min + 3;
    chart.options.scales.x.ticks.max = chart.options.scales.x.ticks.max + 3;

    // Update the chart
    chart.update();
  } else {
    // Chart configuration
    const config = {
      type: 'line',
      data: data,
      options: {
        responsive: true,
        interaction: {
          intersect: false,
          mode: 'index'
        },
        scales: {
          x: {
            display: true,
            title: {
              display: true,
              text: 'Time (seconds)'
            },
	        ticks: {
              stepSize: 3 // Set the step size to 3 seconds
            }				
          },
          y: {
            display: true,
            title: {
              display: true,
              text: 'Recording Size (KBytes/sec)'
            },
            suggestedMin: 0, // Set the suggested minimum value of the y-axis to 0
            suggestedMax: Math.max(...sizeDifferences, 50) // Adjust the maximum vertical size (in KB) as needed
          }
        }
      }
    };

    // Get the chart canvas element
    const chartElement = document.getElementById('chart');

    // Create a new chart instance
    window.chartInstance = new Chart(chartElement, config);
  }
}	

// Function to update the data display
    function updateDataDisplay() {
      const dataDisplayElement = document.getElementById('dataDisplay');
      const latestSize = footageSizes[footageSizes.length - 1];
	  
	  if (footageSizes.length >= 2) {
		const previousSize = footageSizes[footageSizes.length - 2];
		const sizeDifference = latestSize - previousSize;

		const latestTimestamp = timestamps[timestamps.length - 1];
		const previousTimestamp = timestamps[timestamps.length - 2];
		const timeDifference = (latestTimestamp - previousTimestamp) / 1000; // Convert milliseconds to seconds
		const changePerSecond = sizeDifference / timeDifference;

		dataDisplayElement.innerHTML = `Latest SD usage Size: ${latestSize} KBytes<br>
		Size Difference in 3 sec: ${sizeDifference.toFixed(2)} KBytes<br>
		Size Change per Second: ${changePerSecond.toFixed(2)} KBytes/s`;
	  } else {
		dataDisplayElement.innerHTML = `Latest Footage Size: ${latestSize} KBytes`;
	  }
  }
	
    // Fetch used size initially
    fetchUsedSize();

    // Fetch used size every 3 seconds
    setInterval(fetchUsedSize, 3000);
  </script>
</body>
</html>

Here’s an explanation of the code:

The HTML structure consists of a <head> section and a <body> section.
In the <head> section, the page title is set to “IP Camera Recording Size per second”. It also includes references to the jQuery library (jquery-3.7.1.min.js) and the Chart.js library (https://cdn.jsdelivr.net/npm/chart.js).
The CSS styles define the dimensions and appearance of the chart and the data display section.
Inside the <body> section, there’s an <h1> heading element with the title “IP Camera Recording Size per second”.
A <canvas> element with the id “chart” is used to render the line chart.
A <div> element with the id “dataDisplay” is used to display the latest SD card usage information.
The JavaScript code begins by declaring two arrays: timestamps and footageSizes. These arrays will store the timestamps and corresponding footage sizes retrieved from the server.
The fetchUsedSize() function is defined. It uses jQuery’s $.ajax() method to send an HTTP GET request to the /sdusedsize route of the server. Upon success, it extracts the numerical value from the response, updates the timestamps and footageSizes arrays, removes the oldest data if more than 3 minutes of data is present, and finally calls the generateGraph() and updateDataDisplay() functions to update the chart and data display.
The generateGraph() function creates or updates the line chart using the Chart.js library. It calculates the size differences between consecutive footage sizes, determines the starting index based on the length of the data, prepares the data and configuration for the chart, and then creates a new chart instance or updates an existing one.
The updateDataDisplay() function updates the information displayed in the dataDisplay element. It calculates the size difference and change per second based on the latest and previous footage sizes and timestamps.
The initial call to fetchUsedSize() fetches the used size initially.
The setInterval() function is used to call fetchUsedSize() every 3 seconds to continuously update the data and chart.

(D) A Node.js code snippet that sets up a basic Express server in the file “storagesize.js” to handle HTTP requests is created.

const express = require('express');
const app = express();
const port = 3000;
const { exec } = require('child_process');

// Route to get the used size of the SD card in KByte
app.get('/sdusedsize', (req, res) => {
  exec('df /dev/mmcblk0p1 --output=used --block-size=1K', (error, stdout, stderr) => {
    if (error) {
      res.status(500).send('Error retrieving SD card usage');
      return;
    }

    if (stderr) {
      res.status(500).send('Error retrieving SD card usage');
      return;
    }
	console.log(stdout.trim() + ' KBytes');
    res.send(stdout.trim());
  });
});

app.use(express.static('public'));

app.listen(port, () => {
  console.log(`Server is running on port ${port}`);
});

Here’s an explanation of the code:

The code begins by importing the express module and creating an instance of the Express application using express(). This allows us to create routes and handle HTTP requests.
The port variable is set to 3000, specifying that the server will listen on port 3000.
The exec function from the child_process module is imported. This function is used to run shell commands.
The code defines a route using app.get('/sdusedsize', ...). This route is accessed via an HTTP GET request to /sdusedsize.
Inside the route handler function, the exec function is used to execute the shell command 'df /dev/mmcblk0p1 --output=used --block-size=1K'. This command retrieves the used size of the SD card in kilobytes (1K block size) and outputs the result.
If there is an error executing the command or if there is any output in the stderr stream, an error response with a status code of 500 is sent back to the client.
If the command is executed successfully and there is no error or output in stderr, the result is sent back to the client as the response body.
The express.static middleware is used to serve static files from the public directory. This allows you to serve HTML, CSS, JavaScript, and other static files.
The server is started by calling app.listen(port, ...), which listens for incoming HTTP requests on the specified port.
When the server starts successfully, a log message is printed to the console.

Overall, this code sets up a basic Express server with a route to retrieve the used size of an SD card and serves static files from the public directory.

Here is a screen capture of the generated chart. At the 87-second mark, I made a change to the coding method of the IPCAM from H.265 to H.264. As a result, the recording size doubled.

The effect of other factors on recording size can be visualized by this chart as well.

A chart showing recording size of IPCAM using Ajax & Express

Related posts: