NuSphere Forums Forum Index
NuSphere Forums
Reply to topic
Limitation with sleep() & flush() inside loop


Joined: 21 Feb 2011
Posts: 11
Location: Olinda / PE
Reply with quote
I need to code the "server part" of JavaScript's EventSource. The PHP script must contain the correct header content type, an infinite loop to keep the connection opened for some time (stream time) and the output needs to be sent immediately. The following code is the PHP script:

Code:
<?php
header("Content-Type: text/event-stream\n\n");
header("Cache-Control: no-cache");

$currentHitCount = 0;

$timeout = time() + 150;
while (time() < $timeout) {
  $currentHitCount++;
  echo("event: hits\n");
  echo("data: {\"count\": $currentHitCount}\n\n");
  flush();
  sleep(10);
}
echo("event: maxExecutionTime\n");
echo("data: {\"msg\": \"Um máximo de 150 segundos de execução contínua foi atingido. Aguarde 3 segundos\"}\n\n");
?>


As you can see, to satisfy the requisite of "send the response immediately" there are the use of flush() after the echoes. To not load the server with this infinite loop, I use sleep(10) to halt the script execution by 10 seconds.

This script works PERFECTLY on Apache 2.4, outputting the response each 10 seconds and ending the stream after 150 seconds*. On SRV Webserver (debugging or not) there is no output. The loop seems to be correctly debugged (breakpoints on it, works), but no response is received. There are some limitation on SRV WebServer with loops, sleep() and/or flush()? I tried to use the SRV address on external chrome with no success (the same results, ie, no results received)

The html to reproduce the problem is the following:

Code:
<!DOCTYPE html>
<html>
  <head>
    <title>Teste</title>
  </head>
  <body>
    <span id="respostas"></span>
    <script type="text/javascript">
    //<![CDATA[
    if (EventSource) {
      var hits = new EventSource("/zolc/hits.php");
     
      hits.addEventListener("hits", function(event) {
        document.getElementById("respostas").innerHTML = document.getElementById("respostas").innerHTML + "hit = " + JSON.parse(event.data).count + "<br>"; 
      }, false);
     
      hits.addEventListener("maxExecutionTime", function(event) {
        document.getElementById("respostas").innerHTML = document.getElementById("respostas").innerHTML + "msg = " + JSON.parse(event.data).msg + "<br>";
      }, false);
    }
    //]]>
    </script>
  </body>
</html>



* The EventSource will reconect after 3 secons automatically, so I have a constant stream for 150 seconds, one pause of 3 seconds e another 150 seconds indefinitely. This is perfect to me

_________________
CF
View user's profileFind all posts by derekwildstarSend private messageVisit poster's websiteAIM AddressMSN MessengerICQ Number
Site Admin

Joined: 13 Jul 2003
Posts: 8334
Reply with quote
I'm not sure how this script would work under Apache. You didn't turn off output buffering. Flush does not help if OB stuff is in effect. What you need is either ob_end_flush() function call immediately after last header() call or corresponding output_buffering setting in php.ini If script works for you, OB must have been already disabled in php.ini, then why didn't you do the same in php.ini for srv?
Also I'd recommend to set max execution time. Add a set_time_limit(xxx) function call with appropriate limit, such as 300 sec. Default php limit is 30, not enough for running a 150 sec script.

as of SRV, you're correct -- it wasn't designed for such cases, but with a quite simple tweak I got it working with your script (after fixes suggested above).

_________________
The PHP IDE team
View user's profileFind all posts by dmitriSend private messageVisit poster's website


Joined: 21 Feb 2011
Posts: 11
Location: Olinda / PE
Reply with quote
Hello! Thanks for the answer. I will comment separatelly

dmitri wrote:
I'm not sure how this script would work under Apache. You didn't turn off output buffering. Flush does not help if OB stuff is in effect. What you need is either ob_end_flush() function call immediately after last header() call or corresponding output_buffering setting in php.ini If script works for you, OB must have been already disabled in php.ini, then why didn't you do the same in php.ini for srv?


I revised the output_buffering at SRV's php.ini and my local Apache's php.ini and, in fact, the configuration on Apache was Off, while the one on SRV was 4096. My production Apache's php.ini output_buffering is also off, so I will not have problems, I hope. Thank you for this valuable tip!

I disabled the output_buffering on SRV and I checked it with phpinfo(): the output buffering is really OFF. I tried to execute the same example that runs perfectly on my local Apache. The results were disappointed: no output until the end of script. So I followed your tips and added ob_end_flush() immediately after the last header call and i got the same result: no output until the end of script. My script, after your recomendation looks like this:

Code:
<?php
header("Content-Type: text/event-stream\n\n");
header("Cache-Control: no-cache");
ob_end_flush();
// CONFIGURE AQUI //////////////////////////////////////////////////////////////
define("MAX_EXECUTION_TIME",150);
define("SLEEP_TIME",10);
define("MAX_RETRY_INTERVAL",60);
define("MIN_RETRY_INTERVAL",10);
////////////////////////////////////////////////////////////////////////////////

$lastHitCount = 0;
$hitsChanges = 0;
$timeout = time() + MAX_EXECUTION_TIME;
$currentHitCount = 0;

while (time() < $timeout) {
  if (rand(0,1)) {
    $currentHitCount++;
    echo("event: OnNewHits\n");
    echo("data: {\"count\": $currentHitCount}\n\n");
    $lastHitCount = $currentHitCount;
    $hitsChanges++;
    flush();   
  }
  sleep(SLEEP_TIME);
};
// Calculando o tempo de tentativas baseado na quantidade de hits
$retrySec = round((1 - ($hitsChanges / (MAX_EXECUTION_TIME / SLEEP_TIME))) * (MAX_RETRY_INTERVAL - MIN_RETRY_INTERVAL) + MIN_RETRY_INTERVAL);
$retry = $retrySec * 1000;

echo("retry: $retry\n\n");

echo("event: OnMaxExecutionTime\n");
echo("data: {\"hitCount\": $currentHitCount, \"hitsChanges\": $hitsChanges, \"nextRetryIn\": $retrySec }\n\n");
?>


I guessed this would work on any situation (output buffering on or off), but it doesn't work. There are another configuration to turn off? Because, with you, this example works...

Another strange behaviour, which leads me to guess there are some kind of bug with DBG or SRV is that the breakpoints only works on the above file if I keep it since the start of debug (F9). If I put a BP inside the loop and press F9, the program will stop every 10 seconds. This is correct, but if I start the program (F9) and only after this put a breakpoint inside the loop with the program running, this breakpoint will never stop the program.

dmitri wrote:
Also I'd recommend to set max execution time. Add a set_time_limit(xxx) function call with appropriate limit, such as 300 sec. Default php limit is 30, not enough for running a 150 sec script.


Thanks for the recomendation, but this is not necessary anymore. I limited my loop execution to 150 seconds (2.5 minutes). The max execution time of PHP defaults to 300 seconds (5 minutes), so, my script will always ends gracefully. Later I will change it to detect the current Max Execution Time and always exit after half of this timeout has passed. Thank you!

dmitri wrote:
as of SRV, you're correct -- it wasn't designed for such cases, but with a quite simple tweak I got it working with your script (after fixes suggested above).


I do not have lucky using your tips to make SRV run correctly, even when the OB is off inside php.ini. There are something more I have to do? If you need more informations, about my OS, my PHPed version or other thing, feel free to ask

_________________
CF
View user's profileFind all posts by derekwildstarSend private messageVisit poster's websiteAIM AddressMSN MessengerICQ Number
Site Admin

Joined: 13 Jul 2003
Posts: 8334
Reply with quote
Well, if we're talking about SRV that shipped to you, it waits for 4096 bytes output from php then parses the headers, adds missed ones and returns stream to the browser. This is what I changed. Now it does not wait for 4K output buffer. Once the headers are received from php, SRV just returns them and the body (the rest output) to the browser chunk by chunk. It's what you'll get with next update (v18 so far). Meanwhile, if you really need SRV (as of v17 or before) to work with your script, you need to produce 4K dummy output between last header() call and the cycle of the events.

_________________
The PHP IDE team
View user's profileFind all posts by dmitriSend private messageVisit poster's website


Joined: 21 Feb 2011
Posts: 11
Location: Olinda / PE
Reply with quote
dmitri wrote:
Well, if we're talking about SRV that shipped to you, it waits for 4096 bytes output from php then parses the headers, adds missed ones and returns stream to the browser. This is what I changed. Now it does not wait for 4K output buffer. Once the headers are received from php, SRV just returns them and the body (the rest output) to the browser chunk by


Yes, this is the expected behaviour using ob_end_flus(), but this not work. Even when I changed the php.ini i figured that the buffering is active unconditionally

dmitri wrote:
It's what you'll get with next update (v18 so far). Meanwhile, if you really need SRV (as of v17 or before) to work with your script, you need to produce 4K dummy output between last header() call and the cycle of the events.


Ohh... So there are some kind of bug on SRV, good that you (will) fixed it on 18 release. Send 4096 bytes only to satisfy SRV? No, I prefer wait and meanwhile, I will assume my code is correct Smile

Thank you very much Mr. Dimitri. This thread was very useful for me and i hope that it will be useful for others too

_________________
CF
View user's profileFind all posts by derekwildstarSend private messageVisit poster's websiteAIM AddressMSN MessengerICQ Number
Site Admin

Joined: 13 Jul 2003
Posts: 8334
Reply with quote
Quote:
So there are some kind of bug on SRV, good that you (will) fixed

No, it's not a bug. Apache - when it reads from CGI or FCGI process, also waits for some buffer to fill. Whenever you read from a socket, you have to tell what size of buffer you want to read to -- that's how it works and always worked. The difference is that you have php module that runs in Apache process -- it does not use sockets or pipes for communication with php.
And it's not a fix I talked about, just a workaround. It means that in some other cases it may still not work good enough. It's all about a game around two things -- timings and buffer sizes. We definitely can't avoid either.

_________________
The PHP IDE team
View user's profileFind all posts by dmitriSend private messageVisit poster's website


Joined: 21 Feb 2011
Posts: 11
Location: Olinda / PE
Reply with quote
dmitri wrote:
Quote:
So there are some kind of bug on SRV, good that you (will) fixed

No, it's not a bug. Apache - when it reads from CGI or FCGI process, also waits for some buffer to fill. Whenever you read from a socket, you have to tell what size of buffer you want to read to -- that's how it works and always worked. The difference is that you have php module that runs in Apache process -- it does not use sockets or pipes for communication with php.
And it's not a fix I talked about, just a workaround. It means that in some other cases it may still not work good enough. It's all about a game around two things -- timings and buffer sizes. We definitely can't avoid either.


Ah ok... Thank you!

_________________
CF
View user's profileFind all posts by derekwildstarSend private messageVisit poster's websiteAIM AddressMSN MessengerICQ Number


Joined: 21 Feb 2011
Posts: 11
Location: Olinda / PE
Reply with quote
Bad news...

My Apache server on GoDaddy do not works like my local Apache. The output is buffered UNTIL the script ends. I talked with support and there are no workaround for it because one thing: the way how php is installed on Apache is "CGI/FastCGI". Using the phpinfo() I saw that SRV uses the same "Server API", so, all attempts to empty the buffer is inefective, because all output is buffered independently of any PHP configuration.

Take a look at https:// goo.gl / PvGrFE (<-remove the spaces). Go straight to "Apache Configuration" and mod_fastcgi.

Well, I will make the script using simple Ajax Sad

_________________
CF
View user's profileFind all posts by derekwildstarSend private messageVisit poster's websiteAIM AddressMSN MessengerICQ Number
Site Admin

Joined: 13 Jul 2003
Posts: 8334
Reply with quote
did you try to set FcgidOutputBufferSize=0?

_________________
The PHP IDE team
View user's profileFind all posts by dmitriSend private messageVisit poster's website


Joined: 21 Feb 2011
Posts: 11
Location: Olinda / PE
Reply with quote
dmitri wrote:
did you try to set FcgidOutputBufferSize=0?


Hello!

I have no access to httpd.conf file, so I would need to use this on a .htaccess file. This is beyond of my knowledge about Apache configuration. I tried to do some other configurations and I received error 500 from server.

Can you please write here how you could create an .htaccess file with this configuration?

_________________
CF
View user's profileFind all posts by derekwildstarSend private messageVisit poster's websiteAIM AddressMSN MessengerICQ Number
Site Admin

Joined: 13 Jul 2003
Posts: 8334
Reply with quote
if you open https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html#fcgidoutputbuffersize, you'll see that Context: server config, virtual host, meaning the setting can't be changed anywhere else but httpd.conf and the files included in/parsed with this file. No chances .htaccess would help, but you can try to setup local apache with your own settings to check, then, if they prove they helped, you may want to ask godaddy to fix that problem on their site as well. Finally, it makes no sense to buffer the php output in fcgid because it's already buffered in php itself and this buffering is manageable unlike it is in fcgid

_________________
The PHP IDE team
View user's profileFind all posts by dmitriSend private messageVisit poster's website


Joined: 21 Feb 2011
Posts: 11
Location: Olinda / PE
Reply with quote
dmitri wrote:
if you open https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html#fcgidoutputbuffersize, you'll see that Context: server config, virtual host, meaning the setting can't be changed anywhere else but httpd.conf and the files included in/parsed with this file. No chances .htaccess would help, but you can try to setup local apache with your own settings to check, then, if they prove they helped, you may want to ask godaddy to fix that problem on their site as well. Finally, it makes no sense to buffer the php output in fcgid because it's already buffered in php itself and this buffering is manageable unlike it is in fcgid


I totally agree that this buffer behaviour of FCGI is nonsense. Unfortunately my GoDaddy subscription uses a shared host and this type of configuration cannot be done only for me. Well, I guess it could be done via virtual host, but the plan that I signed do not give to me this power ($$$).

Thank you very much Dimitri for all your help. At least we discovered this default (and nonsense) behaviour of FastCGI, and we can now go straight to point, if other similar problem occurs

_________________
CF
View user's profileFind all posts by derekwildstarSend private messageVisit poster's websiteAIM AddressMSN MessengerICQ Number
Limitation with sleep() & flush() inside loop
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
All times are GMT - 5 Hours  
Page 1 of 1  

  
  
 Reply to topic