Child process setInterval sporadically not firing

问题

My application places bets on certain sporting events. Almost like an automatic betting bot. A part of this is to track the current status of the event so it can make an informed calculation whether to place a bet or not. To do this, I poll the status of an event every minute using setInterval. A single sporting event is "watched" by 1 child process. There could be up 100+ sporting events at any one time meaning there could be 100+ child process' spawned & actively polling.

worker/index.js

const logger = require('../lib/logger/winston')
const utils = {
    updateEventState: require('../utils/update-event-state')
}
const db = require('../db')

let eventStatesInterval

module.exports = async function() {
    try {
        logger.info(`[STARTED FOR EVENT ${process.EVENT_ID} (Process: ${process.pid})]`)

        const event = await db.getEvent(process.EVENT_ID)

        logger.info(`[STARTING STATE POLLING FOR EVENT (${process.EVENT_ID} - Process: ${process.pid})]`)

        eventStatesInterval = setInterval(utils.updateEventState, 60000, event) // 1 min

        process.on('exit', code => {
            clearInterval(eventStatesInterval)
        })
    } catch(err) {
        throw err
    }
}

utils/update-event-state.js

const logger = require('../lib/logger/winston')
const db = require('../db')

module.exports = async function(event) {
    try {
        const update = {}

        if (!process.CHECKING_EVENT) {
            process.CHECKING_EVENT = true

            logger.info(`[POLLING STATE (${process.EVENT_ID} - Process: ${process.pid})]`)

            // Some async operations polling APIs to get the full status of an event
            const hasEnded = await api.getHasEventEnded(process.EVENT_ID)

            await db.updateEvent(process.EVENT_ID, update)

            if (hasEnded) {
                process.exit(0)
            }
            process.CHECKING_EVENT = false
        }
    } catch(err) {
        throw err
    }
}

It's also worth noting that a single child process could have more setInterval process' further down the line. For example, if I place a bet that is not fully matched. I poll to check when/if it gets matched. This is also run on a setInterval basis (about every 5 secs). Check ing the logs, some process' are polled correctly every minute but a couple (inconsistent number each time) are not being polled at all. Looking at the logs for a specific process, I get:

For reference, the current time was 22:33 when that screenshot was taken so the interval had not happened in over an hour

There was only 4 events (child process') running at the time.

That is an example screenshot. A process can log several interval callbacks & then just...stop. No more logs at all. No errors or anything. Just stops.

This application runs on a Digital Ocean box with 4GB memory & 2 vCPUs. The app is dockerised. When running docker stats, 100% CPU is being used constantly. When starting with docker run, I limit the memory usage but not the CPU. Could this be the issue? Would setInterval callback liable not be invoked due to CPU constraints?

It's worth noting that this feature to poll the event state is new (5 days) & I had never had an issue with setInterval beforehand (I don't know how much CPU was being used though). I was initially polling the state every 30 seconds & when I noticed this problem. When I checked docker stats, almost 200% CPU usage was shown. Lowering it to 1 minute instead has fixed that slightly. I have the process.CHECKING_EVENT global bool there to not block the thread & not cause a pile up of tasks on the stack.

来源：https://stackoverflow.com/questions/65030139/child-process-setinterval-sporadically-not-firing

标签

javascript

node.js

setinterval

child-process