Jump to content

1 (One) Minute Cron Jobs :)


PauloV

Recommended Posts

Hello blestars :blesta:

 

Long time I post in here eheh ;)

 

My request is simple, and me personally and one client that Im working with, asks for CORE implementation, to be able to set, also, Cron Jobs to run in "Interval", by 1 (one) minute and not by the default minimum 5 minutes, that I think is to hight to check for exemple new Incoming Emails on IMAP/POP Support Manager (Pro also) :P

 

Here is the solution to implement on CORE, only 2 lines of code change in 2 files :blesta:

 

Edit: [your-blesta-instalation]/app/controllers/admin_company_automation.php
Find on line: 160
$intervals = array(5=>5, 10=>10, 15=>15, 30=>30, 45=>45);
Change to:
$intervals = array(1=>1, 5=>5, 10=>10, 15=>15, 30=>30, 45=>45);
Edit: [your-blesta-instalation]/app/controllers/cron.php
Find on lines 2087/2088:
// The last run date rounded down to the nearest 5 minute interval
$last_run_date = date("c", floor($this->Date->toTime($last_run->start_date)/(60*5))*(60*5));
Change to:
// The last run date rounded down to the nearest 1 minute interval
$last_run_date = date("c", floor($this->Date->toTime($last_run->start_date)/(60))*(60));
Finally, change your server/host blesta CRON task from something like:
*/5 * * * * /usr/bin/php /home/[path-to-your-blesta-instalation]/index.php cron
Change it to:
* * * * * /usr/bin/php /home/[path-to-your-blesta-instalation]/index.php cron

Thanks in advance :P

 

Keep it up :)

 

Regards,

PV

 

Link to comment
Share on other sites

Bad bad bad idea, we spoke about this before I believe in this forum and it would cause a server overload if you run a cron every minute because it does everything from orders, to emails, downloading tickets.

 

 

But if you set Only some tasks to one minute like:

 

Download Tickets

Delivery Invoices

Provision Paid Pending Services

 

(I think are the most important tasks to run quickly)

 

The server will not overload, because the outher tasks are "Pending"  and they will not run every one minute :)

 

Or at least have a RULE to apply some "time" limits on some Modules/Plugins this way the adverge admin will not be able to set by mistake the one minute task :) of course this will take more time to implement on core files because it will need changes on several core files, in modules/plugins, and maybe database

Link to comment
Share on other sites

Still don't agree with it, you will run into lots of problems but we'll wait for the guys to explain better developer to developer.

Download tickets works straight away when you use piping as it doesn't use the cron job.

 

 

Yes, PIPE works wonderfull, but wen for exemple you use multi company and/or multiple MX servers you have to use IMAP or POP :)

 

Yes, I whant to see a response from Cody and/or Tyson (Paul also eheh) :P

Link to comment
Share on other sites

the idea is really good PauloV, I agree with you that customers can not wait 5mins to get a service activated when it's paid.

 

In this post I also given how I do make 2 mins cron frequency, with a security to prevent  runing a cron when last one is not yet ended:

 

http://www.blesta.com/forums/index.php?/topic/2990-email-not-being-sent-on-invoice-creation/page-2

 

so with PauloV technique, there is diffrent way, and that fines, but 5 mins wait is too long in some case

Link to comment
Share on other sites

1 minute cron jobs are a bad idea. It's too easy for them to pile up and execute on top of each other.

 

If the point is to pull email from SMTP/IMAP server in near real time, create a script that runs constantly and polls the SMTP/IMAP server every 30 to 60 seconds.

 

For example:

<?php

while (true) {
    $window = 60;
    $start = microtime(true);
    
    // TODO: do stuff

    $runtime = microtime(true) - $start;
    // wait until we can run again...
    sleep(max(0, (int) $window - $runtime));
}
Link to comment
Share on other sites

 

1 minute cron jobs are a bad idea. It's too easy for them to pile up and execute on top of each other.

 

If the point is to pull email from SMTP/IMAP server in near real time, create a script that runs constantly and polls the SMTP/IMAP server every 30 to 60 seconds.

 

For example:

<?php

while (true) {
    $window = 60;
    $start = microtime(true);
    
    // TODO: do stuff

    $runtime = microtime(true) - $start;
    // wait until we can run again...
    sleep(max(0, (int) $window - $runtime));
}

 

 

They only "pile up" if there isnt any verification :) for exemple put a key on database that tell its still running, if so, the cron will execute returning/executing "nothing", to finish and not run in background, and they will not "pile up" :)

 

Isnt the way that is working right now? they check the time/date on the cron database and if is not finish the cron will not run?

Link to comment
Share on other sites

As mentioned above, I use this: http://timkay.com/solo/

 

It's dead simple and it's not posible to have more than one cron job at same time, as cron can not be run untill last cron job ended.

 

Because an open port is directly tied to a running process, chances of inconsistency and having to detect and cleanup orphaned PID files, are zero.

Link to comment
Share on other sites

They only "pile up" if there isnt any verification :) for exemple put a key on database that tell its still running, if so, the cron will execute returning/executing "nothing", to finish and not run in background, and they will not "pile up" :)

 

Isnt the way that is working right now? they check the time/date on the cron database and if is not finish the cron will not run?

 

Yes that's what Blesta does now. I'm not stating that you can't have 1 minute cron jobs that work. I'm just saying, in practice, it's not really a good idea as the risk of a collision is high.

 

 

Plus, why have 1 minute cron jobs when you have can SMTP polling every 15 seconds? B)

Link to comment
Share on other sites

  • 2 months later...

 

1 minute cron jobs are a bad idea. It's too easy for them to pile up and execute on top of each other.

 

If the point is to pull email from SMTP/IMAP server in near real time, create a script that runs constantly and polls the SMTP/IMAP server every 30 to 60 seconds.

 

For example:

<?php

while (true) {
    $window = 60;
    $start = microtime(true);
    
    // TODO: do stuff

    $runtime = microtime(true) - $start;
    // wait until we can run again...
    sleep(max(0, (int) $window - $runtime));
}

 

 

Process monitors (supervisord, daemontools, launchd etc) are a great way to run a looping script like this in the background on a server, just like a daemon process. They allow start/stop/status/start at boot and auto restart on fail. You can avoid cron completely and they are simple to use. You can even use a shell script rather than php to handle the looping part (execute a php script each loop, e.g. /usr/local/bin/php myCronTasks.php).

 

I use one to run multiple and simultaneous "background services", sort of like a single multi-threaded daemon, so tasks like imap, smtp, notifications are near real time. However, there's a few more tricks needed involving a DB or temp files on disk if you want to run multiple doing the same thing at the same time and avoid overlap. If you only need a single service, then php and a process monitor is all you need.

 

There is never a worry about overlap or locking like cron and things just keep on running. If you handle/catch errors properly so the loop never breaks then the service will run forever (use set_time_limit(0) and gc_collect_cycles()).

 

 

I strongly recommend this approach. Use supervisord if you don't mind installing python or daemontools if you don't want any libraries.

 

 

I haven't tried this with Blesta yet. Cody, can you see any issues in simply executing the Blesta cron command inside a loop?

Link to comment
Share on other sites

I haven't tried this with Blesta yet. Cody, can you see any issues in simply executing the Blesta cron command inside a loop?

 

You could create race conditions. Each PHP process is spun off in a new thread, so there's no guarantee one finishes before the next is requested. This is especially problematic when a task performs operations that result in the PHP process blocking for IO, such as processing payments, or receiving email. If you were to do this, you would need to verify that the task ended before starting up the next.

Link to comment
Share on other sites

You could create race conditions. Each PHP process is spun off in a new thread, so there's no guarantee one finishes before the next is requested. This is especially problematic when a task performs operations that result in the PHP process blocking for IO, such as processing payments, or receiving email. If you were to do this, you would need to verify that the task ended before starting up the next.

 

 

Thanks Cody.

 

 

 

I just took a look at /blesta/app/controllers/cron.php, so rather than execute the cron job in the loop, create an instance of the Cron class and do something like this in your loop:

require('YOUR INSTALL DIR/blesta/app/controllers/cron.php');

//re-use
//$cron = new Cron();

//your loop
while(true)
{
    try
    {
        //or new every time
        $cron = new Cron();
        $cron->index();
    }
    catch(Exception $e)
    {
        //log etc
    }
    //your sleep/interval stuff etc
}

Does this class contain everything needed/used by the normal cron job? Should other functions also be called, eg preAction()?

Link to comment
Share on other sites

  • 3 years later...
On 8/23/2019 at 9:27 PM, stacklicense said:

The whole 5 minute cron thing is bizzare, why do orders submitted via the order form even need to wait for a cron job to fire...

Perhaps also look into implementing something similiar to hostbillapp.

https://hostbill.atlassian.net/wiki/spaces/DOCS/pages/491527/HostBill+Queue+Performance

  • Customers may pay with a payment method that takes the client away from Blesta, which posts back to Blesta when the payment is completed.
  • Provisioning services through certain modules may take a significant amount of time to complete, such that making the user wait in the browser may result in a timeout.

We are considering an enterprise level feature that adds a daemon to your server that runs continuously and processes certain jobs like order provisioning in near real-time.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...