Hive response time

I've been navigate around the Hive, and keep hitting delays in getting pages back. It seems to average out to a good 5-count - sometimes longer, it's rarely less than a 2-count though.

Is there something that can be done about this?


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
john's picture

Tools for measuring HTTP server response time

Anybody know of any tools for measuring an HTTP server's response time? I think I want to run it from the client, so that network latency is lumped into the measurement.


jurjen's picture

re: Tools for measuring HTTP server response time

found this article at Drupal.org :

Server Performance Benchmarks


tamhas's picture

re: Tools for measuring HTTP server response time

At 09:24 AM 3/1/2007, jurjen wrote:

>found this article at Drupal.org :
>
>Server Performance Benchmarks

Looks like a number of things to try.

=======================================================================
Consulting in Architecture-Driven Modernization and Transformation
-----------------------------------------------------------------------
Thomas Mercer-Hursh, Ph.D. email: thomas@cintegrity.com
Computing Integrity, Inc. voice: 510-233-5400
60 Belvedere Avenue fax: 510-233-5446
Point Richmond, CA 94801-4023 url: http://www.cintegrity.com
=======================================================================


jurjen's picture

bad index usage

phpMyAdmin shows, in its status view, several values that indicate bad index usage.

I have now added an index on table og_uid, on fields (uid,nid).
This table belongs to the OG module which is not a default Drupal module, which confirms my earlier suspicion.

It seems like a good idea to analyze many more queries, especially in the OG module first and then in other non-default modules.


tamhas's picture

Re: bad index usage

Here is another example from the Recent page.

387.86 1
drupal_lookup_path
SELECT COUNT(pid) FROM url_alias

Now, this is a table scan inherently, but there is only a handful of
records in that table.
Refresh the page and the value is 1.37.

Now, I don't know how MySQL works compared to Progress, but the
second query could well be being fulfilled out of the -B equivalent,
making it a lot faster, but the value for the initial one is still
remarkably high.

From Recent:
831.04 1
user_access
SELECT DISTINCT(p.perm) FROM role r INNER JOIN permission p ON p.rid
= r.rid WHERE r.rid IN (2,3,4)

From Site Map:
607.07 1
user_access
SELECT DISTINCT(p.perm) FROM role r INNER JOIN permission p ON p.rid
= r.rid WHERE r.rid IN (2,3,4)

then a refresh and it is under 1 ms!

=======================================================================
Consulting in Architecture-Driven Modernization and Transformation
-----------------------------------------------------------------------
Thomas Mercer-Hursh, Ph.D. email: thomas@cintegrity.com
Computing Integrity, Inc. voice: 510-233-5400
60 Belvedere Avenue fax: 510-233-5446
Point Richmond, CA 94801-4023 url: http://www.cintegrity.com
=======================================================================


john's picture

SIFR headers and performance

Offline, Jurjen and I were discussing whether the SIFR headers have an impact on performance. He removed them in staging, and asked if I thought staging was any faster than live. My reply:

It's hard to say. The response times are so wildly variable that it's hard to get a feel for whether one is faster than the other.

I do notice that the SIFR headers appear about a half second after the browser (Opera) is done downloading and displaying the rest of the page contents.

We could try clobbering the SIFR (except for the SiteName) in live for now, and see how we like it.


tamhas's picture

SIFR

I just tried a couple examples in staging, notably site map and recent, which are two tabs I use a lot and which often seem slow ... and which are good candidates for poor query performance. A couple of these in staging were satisfyingly quick, but one of the clicks on Recent there got me the longest delay that I have seen in a long, long time. So, I don't think it is the SIFR.


jurjen's picture

Re: SIFR

Funny, since sitemap and recent both don't use SIFR headers :-)

But I am afraid I have to agree that SIFR is not the performance bottleneck.

It is hard to imagine that Drupal is by default slow, the software is used by many, many websites. Could it be one of the extra modules that we have installed on top of Drupal? Or perhaps some feature that we are using but that is not used on most other websites?


john's picture

Re: SIFR

I still think that we are at the mercy of wildly variable response times because of the fact that we are on a shared server. For example, I just got an 8 second load time for another site's page (hosted on WestHost) which is not Drupal, but just a simple HTML page.

Joanju (hosted on 1and1) seems to be a lot more consistent about giving me 2 second page refresh times.

Are there any Apache experts out there who could give us pointers about gathering measurements for page response times?


jurjen's picture

shared server

Well, every server is a shared server (unless it is a dedicated server) but what we have here at Westhost is a dedicated server in a virtual machine that runs on a shared server.
Would be nice if they were willing to move our virtual machine to a different (quiet) box and see if it makes a difference.


tamhas's picture

Which pages

One of the big questions is whether this is happening randomly on all pages or whether it happens to some pages in particular. E.g., I just got a very long delay refreshing this page.

I am pretty sure it isn't completely consistent, though, since the pages I associate most with long delays sometimes come up quickly as well.


jurjen's picture

cron, perhaps?

Cron fires every 8 minutes to update the search index and to do some other tasks, like importing email into comments.
Could that be causing peak loads during which we see long refresh times?


john's picture

Re: cron, perhaps?

That's a good question. Perhaps we should look into gathering system-wide measurements, rather than just staying focused on the web server response times.


john's picture

Response from WestHost

This is the response I received from WestHost:


I have tested 4 pages on this account over the last 4 days. The average load time was 2.02 seconds. The server was within acceptable ranges. The apache benchmark tests I ran which excluded any network traffic issues, testing the responsivness of the server was very fast. All results of my tests are below.

Average = 2.02

Feb 26
URL:  http://oehive.org/
Measured Speed (using 10 mbps T3 connection): 1.64 sec

URL:  http://www.oehive.org/taxonomy/term/131/
Measured Speed (using 10 mbps T3 connection): 1.72 sec

URL:  http://www.oehive.org/taxonomy/term/214/
Measured Speed (using 10 mbps T3 connection): 2.61 sec

URL:  http://www.oehive.org/taxonomy/term/128/
Measured Speed (using 10 mbps T3 connection): 1.41 sec


Feb 25th
URL:  http://oehive.org/
Measured Speed (using 10 mbps T3 connection): 2.09 sec

URL:  http://www.oehive.org/taxonomy/term/131/
Measured Speed (using 10 mbps T3 connection): 2.36 sec

URL:  http://www.oehive.org/taxonomy/term/214/
Measured Speed (using 10 mbps T3 connection): 2.08 sec

URL:  http://www.oehive.org/taxonomy/term/128/
Measured Speed (using 10 mbps T3 connection): 2.14 sec


Febuary 24rth

Speed test results:
URL:  http://oehive.org
Measured Speed (using 10 mbps T3 connection): 3.09 sec

URL:  http://www.oehive.org/taxonomy/term/214
Measured Speed (using 10 mbps T3 connection): 1.88 sec

Speed test results:
URL:  http://www.oehive.org/taxonomy/term/131
Measured Speed (using 10 mbps T3 connection): 1.84 sec

Speed test results:
URL:  http://www.oehive.org/taxonomy/term/128
Measured Speed (using 10 mbps T3 connection): 1.41 sec


$ ab -n20 http://oehive.org/
This is ApacheBench, Version 1.3d <$Revision: 1.67 $> apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking oehive.org (be patient).....done
Server Software:        Apache/1.3.27
Server Hostname:        oehive.org
Server Port:            80

Document Path:          /
Document Length:        297 bytes

Concurrency Level:      1
Time taken for tests:   0.032 seconds
Complete requests:      20
Failed requests:        0
Broken pipe errors:     0
Non-2xx responses:      20
Total transferred:      10460 bytes
HTML transferred:       5940 bytes
Requests per second:    625.00 [#/sec] (mean)
Time per request:       1.60 [ms] (mean)
Time per request:       1.60 [ms] (mean, across all concurrent requests)
Transfer rate:          326.88 [Kbytes/sec] received

Connnection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0     0    0.0      0     1
Processing:     1     1    0.0      1     1
Waiting:        0     1    0.0      1     1
Total:          1     1    0.0      1     2

jurjen's picture

re: Hive response time

I think you count faster than I do :-) Just kidding, I agree that performance is not good.

Not sure but I think that pages with a "book outline" block are slower than others. Perhaps the sql query for the book outline is too complicated or badly indexed. I have seen the code; it sure is too complicated for me! But I am a sql and php newbie. If someone else wants to have a look... be my guest.

Next. Pages that are actually lists of other pages are not fast. Perhaps because those pages show a bunch of headers, which are rendered in that funny orange/red font. I can imagine that this renedering takes some time, if so then we may have to consider a normal font.

Finally, I would like to upgrade to Drupal 5.1 asap because it has several performance improvements. Problem is that some of the extra modules are not ready for 5.1 yet, we have to wait for them.


john's picture

Re: Hive response time

Hi Tim,

We use Drupal with page caching enabled. Because the same page seems to sometimes load fast and other times load slow, I don't think Drupal is to blame.

We use WestHost with a virtual private server. (With zero budget, that was all I was willing to go out-of-pocket for.) I agree that sometimes there is a 5 second page load time, and it is not acceptable. I'll send a complaint to WestHost.

Cheers,
John


jurjen's picture

Hosting provider

I hope Westhost can help, but actually I think it is the php code in Drupal or its MySQL database. Our subversion server is very fast, much faster than the one at sourceforge!! That was a big surprise for me while I was working on Prolint, no reason to have a local repository for speed (Tim?). The subversion browser (websvn) is also fast enough, and phpMyAdmin behaves ok.

Anyway, if the Westhost admins can pinpoint where the problem is and tweak some, that would be great.


Websvn seems ok, but the

Websvn seems ok, but the response to other page navigation has been spotty - there's been a little bit where it was ok, but most of the time the delays between hitting a link and the page load is too long.