Printable View
Anyone else experience increased incidence of this?
The past week has worsened. Typically this occurs if I hit the winkie little blue arrow and attempt direct access to a recent post; sometimes an arduous circuitous pathing will get me there, other times not.
I guess Ian has just shat my nest and I'll live with it... :baby:
Yep! It is annoying. It seems to clear up after a few minutes.
Widget
you'll see this if the ISP load balancing isn't right. Meaning they are trying to feed too many people off of too little bandwidth, the connection slows and PCBD errors arise.
Sometimes this is because there was an issue at the ISP backbone - some T3 connections got dropped - and the ISP compensated by feeding everyone off of the remainder or the lesser bandwidth failover circuits.
Sometimes it is because the ISP either has the wrong equipment, or has not spread out its' bandwidth load evenly across all its' input/output circuits - IE load balancing.
Uh-OH... Now you've done it... :duck:
I get it too, with very recent (minutes) posts. I come back later, and it's O.K.....
[Unless it's been deleted already, of course. :p]
'Here too - it seems to have mainly started in the last few days. I don't recall ever even seeing it before that...
John
:) All of the above, ???
We had this happen to us on the network I run (used to run - new job next week :D :bouncy: .) We ended up changing cisco routers and life got better for our users.
It won't let me see my OWN posts when I make them, sometimes.
This is VERY, VERY bad. :D
I have found a few threads that I could not open with this response.
Lets hope it is not a corrupt database.....
To answer the above:Quote:
Originally Posted by yggdrasil
"Ok, I checked all 82 out of the 84 tables that could be checked. All came out with 'status OK'.
2 sessions tables are in memory, and can't be checked. These have transient data.
It's easy to run the checks, feel free to ask any time.
Ann"
Thank you ANN!
Hi, Ann!! :wave:
Bo,Quote:
Originally Posted by boputnam
I don't fully understand your technical terminology, however, DNS complaints could be your PC, DNS server at work, or your ISP's DNS server if from home, Internet congestion practically anywhere, or (n1.csd.net and ns2.csd.net) simultaneously ?? (not likely). A traceroute to 204.181.152.51 while it is happening would be helpful - ie. a map because someone or something is lost.
J.
so in short, when this occurs, you'd like a tracert to see where the hop is with the longest delay.
These are Ping and tracert results during a PCBD episode.
Oddly, they don't look bad, just that the server web page can't be displayed.
Pinging audioheritage.org [198.80.11.57] with 32 bytes of data:
Reply from 198.80.11.57: bytes=32 time=111ms TTL=48
Reply from 198.80.11.57: bytes=32 time=105ms TTL=48
Reply from 198.80.11.57: bytes=32 time=111ms TTL=48
Reply from 198.80.11.57: bytes=32 time=110ms TTL=48
Ping statistics for 198.80.11.57:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 105ms, Maximum = 111ms, Average = 109ms
Tracing route to audioheritage.org [198.80.11.57]
over a maximum of 30 hops:
1 9 ms 9 ms 10 ms 10.101.176.1
2 9 ms 11 ms 10 ms pos6-0-0.syrcnypth-rtr01.nyroc.rr.com [24.92.227
.94]
3 9 ms 13 ms 11 ms drh-rtr-lph.nyroc.rr.com [24.92.225.25]
4 13 ms 13 ms 13 ms srp2-0.syrcnyspp-rtr04.nyroc.rr.com [24.92.227.2
17]
5 12 ms 14 ms 15 ms srp10-0.syrcnyspp-rtr01.nyroc.rr.com [24.92.224.
137]
6 12 ms 13 ms 14 ms son2-0-1.syrcnyspp-rtr06.nyroc.rr.com [24.92.224
.21]
7 31 ms 29 ms 30 ms so-2-0-0-0.gar1.Philadelphia1.Level3.net [4.78.1
48.1]
8 30 ms 30 ms 105 ms so-7-0-0.mp2.Philadelphia1.Level3.net [209.247.9
.253]
9 32 ms 32 ms 32 ms ae-0-0.bbr1.NewYork1.Level3.net [64.159.1.41]
10 32 ms 26 ms 39 ms ge-6-0-0-53.gar4.NewYork1.Level3.net [4.68.97.69
]
11 33 ms 33 ms 31 ms uunet-level3-oc48.NewYork1.Level3.net [209.244.1
60.182]
12 26 ms 28 ms 27 ms 0.so-6-0-0.XL2.NYC4.ALTER.NET [152.63.21.82]
13 33 ms 33 ms 33 ms 0.so-2-0-0.TL2.NYC8.ALTER.NET [152.63.0.185]
14 56 ms 55 ms 55 ms 0.so-7-1-0.TL2.STL3.ALTER.NET [152.63.0.206]
15 107 ms 107 ms 108 ms 0.so-4-1-0.XL2.DEN4.ALTER.NET [152.63.92.238]
16 106 ms 107 ms 107 ms 178.ATM7-0.GW4.DEN4.ALTER.NET [152.63.93.205]
17 110 ms 107 ms 109 ms csd-gw.customer.ALTER.NET [157.130.160.62]
18 110 ms 106 ms 106 ms apollo.csdco.com [198.80.11.57]
Whether it reaches the destination would be the first consideration, but then the problem could still be that you can reach it but your DNS server can't.Quote:
Originally Posted by hector.murray
There's something screwy going on. I constantly get those messages and have to hit refresh or try a link again to get where I want to go. Even managing attachments comes up with the error.
Ditto !Quote:
There's something screwy going on. I constantly get those messages and have to hit refresh or try a link again to get where I want to go. Even managing attachments comes up with the error.
- Is there a "tracert" that runs from a Mac? If so , what & how do I go about executing this trace ?
Earl,Quote:
Originally Posted by Earl K
If you're using Mac OS X (if not, why not? :confused: ), then you can go to Applications/Utilities/Network Utility and open it up. Click on the Traceroute tab and put in the network address.
I used 198.80.11.57 to get to the audioheritage.org site and got this:
Traceroute has started ...
traceroute to 198.80.11.57 (198.80.11.57), 30 hops max, 40 byte packets
1 192.168.1.1 (192.168.1.1) 3.788 ms 1.538 ms 1.002 ms
2 192.168.0.1 (192.168.0.1) 1.565 ms 1.481 ms 1.439 ms
3 adsl-69-226-47-254.dsl.irvnca.pacbell.net (69.226.47.254) 12.361 ms 11.019 ms 11.143 ms
4 dist1-vlan50.irvnca.pbi.net (67.114.48.66) 18.585 ms 14.046 ms 29.505 ms
5 bb1-g1-1-1.irvnca.sbcglobal.net (67.114.49.51) 14.686 ms 34.848 ms 18.358 ms
6 core2-p5-0.cranca.sbcglobal.net (151.164.41.18) 12.119 ms 19.701 ms 11.96 ms
7 core1-p1-0.cranca.sbcglobal.net (151.164.241.221) 13.144 ms 19 ms 18.806 ms
8 bb1-p8-0.cranca.sbcglobal.net (151.164.40.94) 28.366 ms 15.11 ms 12.637 ms
9 ex1-p15-0.eqlaca.sbcglobal.net (151.164.41.29) 20.816 ms 27.575 ms 21.239 ms
10 sl-st20-la-4-0.sprintlink.net (144.232.154.229) 27.851 ms 13.486 ms 14.399 ms
11 sl-bb21-ana-13-0.sprintlink.net (144.232.20.66) 14.751 ms 15.011 ms 16.204 ms
12 sl-bb23-ana-13-0.sprintlink.net (144.232.1.186) 14.715 ms 14.549 ms 14.158 ms
13 sl-bb25-sj-9-0.sprintlink.net (144.232.20.159) 26.228 ms 28.128 ms 25.727 ms
14 sl-bb23-sj-15-0.sprintlink.net (144.232.3.249) 25.996 ms 24.919 ms 26.204 ms
15 sl-bb21-tac-13-0.sprintlink.net (144.232.20.105) 48.932 ms 46.99 ms 50.987 ms
16 sl-bb20-che-5-1.sprintlink.net (144.232.18.9) 88.7 ms 85.545 ms 84.612 ms
17 sl-gw11-che-9-0.sprintlink.net (144.232.15.150) 86.007 ms 83.669 ms 84.308 ms
18 sl-csd-11-0.sprintlink.net (160.81.226.14) 93.676 ms 89.35 ms *
19 apollo.csdco.com (198.80.11.57) 89.783 ms 95.959 ms 94.473 ms
The response time via Sprint appears to be better than UUnet.
I think there may be a vBulletin bug and maybe Don could check the list of problems for anything that fits.
Several times when I tried to open this thread, nothing happened.
Earl,
Sorry to do this backwards, but you can also ping from the Network Utility by clicking on the Ping tab and entering the location you want to ping.
So putting in www.audioheritage.org got me this:
Ping has started ...
PING www.audioheritage.org (198.80.11.57): 56 data bytes
64 bytes from 198.80.11.57: icmp_seq=0 ttl=46 time=92.409 ms
64 bytes from 198.80.11.57: icmp_seq=1 ttl=46 time=90.8 ms
64 bytes from 198.80.11.57: icmp_seq=2 ttl=46 time=98.574 ms
64 bytes from 198.80.11.57: icmp_seq=3 ttl=46 time=90.511 ms
64 bytes from 198.80.11.57: icmp_seq=4 ttl=46 time=150.317 ms
64 bytes from 198.80.11.57: icmp_seq=5 ttl=46 time=91.536 ms
64 bytes from 198.80.11.57: icmp_seq=6 ttl=46 time=90.829 ms
You can see in the PING line that it reveals the 198.80.11.57 real address.
Yea, I don't understand my terminology, either!! :rotfl:Quote:
Originally Posted by John Nebel
Sorry, John - missed this reply until now.
When I hit that blue arrow taking me to the last post of a Thread, I was getting that DNS error. In some instances, I could go back the the Root (LHF), and then work into the specific Thread and post - but not always. A few times, anywhere I try to go on the Forum is "DNS error".
At the next instance, I will do a trace and post the report.
Thanks Titanium Dome !
I'm still using 9.2 since my computer is only a G3. :eek:
:cheers:
There is likely a PHP or vBulletin bug. Did this start about the time the new photo gallery was put into operation?
Just guessing John, (since I have such a bad short-term memory) , but I think it was approx. 6 to 10 days ago.
- That's as narrow a time frame as I can muster up. :D
Earl,
On Mac OS X referencing an earlier post of yours..
"Mac OS X Version 10.3 requires a Macintosh with a PowerPC G3, G4, or G5 processor, built-in USB; at least 128MB of physical RAM and a built-in display or a display connected to an Apple-supplied video card supported by your computer. Mac OS X does not support processor upgrade cards. Verify your hardware is supported from the list below."
http://www.apple.com/macosx/upgrade/requirements.html
John
Yes. I don't know when the "tinkering" started, but my awareness of it was nearly coincident. I wondered that, myself.Quote:
Originally Posted by John Nebel
Bo,
Displaying this thread directly with
http://audioheritage.csdco.com/vbull...ead.php?t=4987
sometimes returns a blank page. It looks like the PHP script is crashing.
The DNS problem is separate which would make things pretty confusing.
John
Especially for me!! :o:Quote:
Originally Posted by John Nebel
But the error originally came back as: "Cannot find server or DNS error." - see the starting post - that's why I reported it so. Believe me, I'm merely (poorly) reporting the incidence...
I agree with John (since we're troubleshooting by committe :D ) in that it looks like php errors vice DNS. I keep multiple pages of the forum open and only experiance this issue on individual threads, not the whole site.
It appears two or more people can't hit the same post/thread at the same time. I just posted and now cannot access my post at all. Although I got the error the instant I hit Submit Reply, it appears my post still went through.
Ok, after waiting a bit I can now access the post.
Computers are so unbelievably stupid. They make fairly decent game platforms though. :rotfl:
I believe I first noticed it late in Feb. It does seem to be getting worse.Quote:
Originally Posted by John Nebel
Widget
I don't get a DNS error, instead I get an "HTTP 500 Internal Server Error". When I turn off the default IE "friendly error messages", I get the following detailed error message:
Server Error
The following error occurred: [code=SERVER_RESPONSE_CLOSE] The server closed the connection while reading the response. Contact your system administrator.
<HR>Please contact the administrator.
As with others, it only occurs with specific threads and I don't get errors accessing the main forum page. For me, I just started noticing this in the past week or two. I do know that Ann upgraded PHP in this time as part of getting the new photo gallery up and running and it may be related to this (I hope not).
Don
Best wishes to Ian and Ann regarding the correction of this error. I know you're working to get the issue resolved.
Don,Quote:
Originally Posted by Don McRitchie
PHP isn't writing out anything but the page header for certain threads, but not always failing on those threads. This thread is an example as it only let me access it by going directly to the last post. Would you check the vBulletin site for problem reports and patches? I don't have my password to that site handy. Too many d**n passwords.
Ann can look into the PHP side.
John
The forum has been updated to the most current version and patches for the past couple of weeks. The vBulletin forum does not report any recent incidents similar to what we are experiencing. Doing a search on that forum for "internal server error" resulted in this reply:
This error is most likely from an apache module called mod_security, which severly limits vBulletins operation. Your host will either need to configure mod_security to allow vBulletins operation or remove it.
It wouldn't work at all in that case.Quote:
Originally Posted by Don McRitchie
OK - I just hit the "Back" button in my browser from the post above this and got the "page not Found..." error and the link was http://www.audioheritage.org/vbullet...isplay.php?f=7John
'Any progress on this? It's still happening to me several times a day.
John
John Nebel
I don't know if this means anything, but I started getting the same error on a testbed forum on the lansingheritage.org domain. What's interesting is that this forum currently has only two members and no posts. I was the only one on line on that forum and I still got the "internal server error".
Don
Yea, well now the "problem" has occurred trying to Reply.
Got the reply all typed-in, ready to go and WHACK! I tried it twice over the course of a half-hour, same Thread (Timing Alignment) and no-one else seemed to be doing anything there. Pretty discouraging, because the reply text is lost...
Geez Bo, your post would have started a flame war anyway...
This problem, I am back on this thread topic, is becoming a PITA. I have blown off reading and responding a couple of times... on second thought that may be for the better. :D
Widget
Could always try the old PHP and see if that fixes the problem.
http://bugs.php.net/bugstats.php lists php bugs.
Actually something is crashing
[Mon Mar 14 14:14:59 2005] [notice] child pid 259355 exit signal Arithmetic exception (8)
[Mon Mar 14 14:15:00 2005] [notice] child pid 261324 exit signal Arithmetic exception (8)
[Mon Mar 14 14:15:02 2005] [notice] child pid 182582 exit signal Arithmetic exception (8)
Geez Widget, your post would have started a flame war anyway...Quote:
Originally Posted by Mr. Widget
The thread crashes started:
[Sat Feb 19 21:19:07 2005] [error] (54)Connection reset by peer: getsockname
[Mon Feb 21 14:16:58 2005] [notice] caught SIGTERM, shutting down
[Mon Feb 21 14:17:08 2005] [notice] Apache/1.3.26 (Unix) PHP/4.3.10 FrontPage/5.0.2.2634 configured -- resuming normal operations
[Mon Feb 21 14:17:08 2005] [notice] Accept mutex: flock (Default: flock)
[Mon Feb 21 14:22:20 2005] [notice] child pid 197085 exit signal Arithmetic exception (8)
[Mon Feb 21 14:22:31 2005] [notice] child pid 337070 exit signal Arithmetic exception (8)
[Mon Feb 21 14:35:57 2005] [notice] child pid 329307 exit signal Arithmetic exception (8)
[Mon Feb 21 14:49:27 2005] [notice] child pid 336450 exit signal Arithmetic exception (8)
I fixed your log John -
Quote:
Originally Posted by John Nebel
Thanks :)
The crashes started immediately after PHP 4.3.10 was installed
http://test.audioheritage.org/anntestphpinfo.php
Hey, John...
I inadvertently found a seemingly significant difference: I am having trouble directly "quoting" on a reply - it fails to the "white death" page.
However, if I:
- hit the quote button
- type my reply and then
- copy <CTRL-C> my entire quoted reply, and then
- go back out to the "LHF" main page and work inwards
then I am successful in posting using an ordinary "Reply" and pasting-in <CTRL-V> what I copied.
Maybe this is helpful in sussing the error...?
Bo,
It can likely be any php code in vBulletin crashing when the conditions are right. The php interpreter or the built in optimizer with the improbable name "zend engine" are crashing the apache process in whose context they are executing.
Just like the trouble with medicine - naming the problem is not fixing it.
I can tail -f the error log and see the crash message appear then the vBulletin php task crashes
[Tue Mar 15 06:06:35 2005] [notice] child pid 93378 exit signal Arithmetic exception (8)
[Tue Mar 15 06:06:39 2005] [notice] child pid 62438 exit signal Arithmetic exception (8)
[Tue Mar 15 06:06:41 2005] [notice] child pid 39887 exit signal Arithmetic exception (8)
[Tue Mar 15 06:07:31 2005] [notice] child pid 93529 exit signal Arithmetic exception (8)
At least we know where to look to see when the problem is fixed -
/usr/local/apache-fp/logs/error_log, and what the overall cause is.
There are 32K+ documented bugs in php, but only :) 308 in version 4.3.10. Hopefully this one is documented.
John
Hi John
Just as background, Ann installed the latest version of PHP at my request in order to get the new photo gallery up and running. The specific requirements for that gallery are PHP 4.0.2 or newer, MySQL 3.2.3 or newer and GD 2.x.x or Image Magick. My preference was for GD 2.x.x over Image Magic since Vbulletin has a bot spamming prevention feature that requires GD 2.x.x and I thought that installing it would kill two birds with one stone.
To my knowledge, the version of MySQL you had installed was current enough for Photopost and not changed. I am not 100% sure of the following, but I think the previously installed version of PHP was also new enough to meet the Photopost requirement. The problem was that GD was not installed. I believe Ann went to the latest version of PHP because it included GD 2.x.x and this appeared to be the easiest way to gain that functionality.
Therefore, there may be the option of rolling back to an earlier version of PHP and installing GD separately as a means of addressing the current problems.
Don
Don,
Thanks for the info. Ann and I talked about this when she was installing php 4.3.10. It will likely end up working OK.
Fortunately the problem can be recreated frequently enough. On the scale of very hard to find, very seldom to totally broken, it is comfortably to the right.
John
PS
"The PHP Development Team would like to announce the immediate release of PHP 4.3.10 and PHP 5.0.3. These are maintenance releases that in addition to non-critical bug fixes address several very serious security issues. All Users of PHP are strongly encouraged to upgrade to one of these releases as soon as possible."
:rotfl:Quote:
Originally Posted by John Nebel
FYI: Ann has been working on this for days.
Latest: Replace MySQL with another version & replace php with another version.
That needs a another version of gcc which can only be compiled in an earlier version of itself which if one goes back far enough can be compiled with the c compiler distributed with the operating system.
The operating system's c compiler has the nasty habit of insisting that programs be syntactically correct which open source software typically isn't.
Still happening as of 5:50 Pacific 3/16/05... :(
John
Odd, since most of us, nor progammers, aren't, neither... :no:Quote:
Originally Posted by John Nebel
Good one, Bo.
Just to put this in perspective, the level of service that John and Ann are providing on this issue is unparalled. Those that have been with us long enough may remember the previous service provided by a commercial ISP whenever we had forum problems. On no less than three occaisions, our previous forum (provided by the ISP) crashed causing data loss. Not only would the ISP not take any responsibility for their failed software, but they could not provide recent backups and charged for months old data to be retrieved from tapes. When our last commercial forum package begain having resource issues, instead of trying to diagnose the problem and develop a solution, they just shut down our forum and barred us from using it again.Quote:
FYI: Ann has been working on this for days.
Latest: Replace MySQL with another version & replace php with another version.
Since John has taken on hosting our site at no cost, our forum has quadrupled in size, quadrupled in bandwidth, added the new photo gallery and all without any decrease in performance. Regarding data integritry, I think we have lost a grand total of three posts, all due to stupidity on my part in inadvertantly deleting valid posts as part of ongoing maintenance. In one case they restored an entire forum in less than 12 hours with no data loss after an operator error (me).
The current problem will still take some time to resolve as indicated by the scope of work that John and Ann are undertaking. The only reason the problem occured was because John and Ann were working to enhance our forum with the new photo gallery feature. With new, untried software came the risk of bugs and it would appear that these risks have been realized. I would ask that everyone maintain their patience as this issue is worked out.
Regards
Don
P.S. As further evidence of growth, we appear to be days away from having our main page counter roll over to 400,000!
It isn't easy being in their shoes.Quote:
the level of service that John and Ann are providing on this issue is unparalled
I, as I am sure many of us here do, have faith that they will correct the issue, and believe that they have been working diligently on our behalf.
As a matter of fact, I made a donation based on my belief that they will resolve the issue.
I'm not bitching, I just thought that this might help, it is what I am seeing this morning when I look at the photo gallery.
If dumping the Photo Gallery solved the problems and would allow John and Ann to get some sleep, I'd say it really isn't worth the headaches
Widget
Greetings, folks. Ann the technician here. :) Thanks for the kind words, Don McR and all.
Thanks, Don C, for pointing out those php error messages. The reason you're seeing them now is that I turned on a noisy message setting recently. These particular ones turn out to be normal for the photopost app, and I don't think they are related to our problem. I have a local log that stores the same messages with occurance times, and the times don't correlate with when our errors happen. You'll also see this message, which is also normal and doesn't correlate:
PHP Notice: Undefined variable: argv in /audio-heritage/WWW/photopost/index.php on line 15
I'll probably turn off the print-to-screen message setting in php fairly soon, since the messages seem to match what I have in my local log, so maybe no new information here, and they're puzzling to receive during normal forum business.
Still looking for a solution! :)
Best,
Ann
Ann put in php 4.3.9 and the thread crashes have stopped.
[Fri Mar 18 13:09:56 2005] [notice] Apache/1.3.26 (Unix) PHP/4.3.9 FrontPage/5.0.2.2634 configured -- resuming normal operations
[Fri Mar 18 13:09:56 2005] [notice] Accept mutex: flock (Default: flock)
So far, so good...
Thanks for all the effort you've put into tracking this down!
John