WORK WITH FORESIGHT. INCREASE PERFORMANCE.
PROCESS LOAD OPTIMISATION FOR LOBSTER_DATA.
The Process Load Optimisation (PLO) module makes everyday work easier by handling high process loads in an intelligent way. It is a high-performance addition to the Lobster_data integration software when many smaller files need to be converted, the Lobster integration server is running natively or virtually on a server with multiple processor cores, and/or load peaks lead to longer queues.
KEEP THREADS OPEN. REDUCE OVERHEADS
Setting a file opens a thread in Lobster_data, followed by mapping and the closing of the thread. This results in an overhead of approx. 120 milliseconds for each profile run. While this increased effort is negligible when the volume is low, it costs an enormous amount of time when there are ten million jobs per day. By enabling process load optimisation, the Lobster_data PLO module consequently works ahead of many small and medium conversions and simply keeps the thread open for all subsequent mappings. This saves the time required to open the thread and reduces the overhead to just about three milliseconds per profile run. A significant increase in performance, especially during peak loads with numerous jobs running in parallel. The PLO module enables this acceleration through additional intelligent algorithms for forward-looking thread pooling. The result of customer-oriented software development and continuous testing at Lobster.
MODULE ACTIVATION ON RESTART
Load peaks often occur due to a very large number of HTTP requests made at very short intervals. At high loads, the web server holds many queries in the task queue, since the jobs are only processed successively due to the limited, maximum available memory capacity. Thanks to faster thread processing per HTTP request, process load optimisation reduces response time in Lobster_data. HTTP requests are not rejected by the web server – the entire system remains reliably in operation. The PLO module is also a requirement for enabling and optimally using the MongoDB service as a logging server in Lobster_data. If it is omitted, the advantages of this NoSQL database with its significantly improved performance for the persistence of job-related logs and protocols cannot be truly realized. Another plus of the Lobster_data PLO module: no additional configuration is required for its activation. It is done directly via a license extension, so that after a restart of Lobster_data, the change is simply applied and the module is activated.
More articles on our blog.
