FUDforum
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » Imported messages » comp.lang.php » problem with link checker
Show: Today's Messages :: Unread Messages :: Show Polls :: Message Navigator
| Subscribe to topic | Bookmark topic 
Switch to threaded view of this topic Create a new topic Submit Reply
problem with link checker [message #179309] Wed, 03 October 2012 05:46 Go to next message
Helmut is currently offline  Helmut
Messages: 1
Registered: October 2012
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Hello,
I have written a php script whose function is to detect dead links on
a website.
unfortunately,(obviously) my ISP redirects invalid urls (or part of them) to a search site, so I get status 302 instead of 404.
Even worse, in one case an invalid url (http://www.ajrccm.com)
is also redirected to a search site (I don't know by whom) and I get
the status 200 !
Any idea how to solve this problem?

thanx for your help, Helmut
Re: problem with link checker [message #179310 is a reply to message #179309] Wed, 03 October 2012 05:48 Go to previous messageGo to next message
Captain Paralytic is currently offline  Captain Paralytic
Messages: 204
Registered: September 2010
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
On Oct 3, 10:46 am, Helmut <helmut_bl...@web.de> wrote:
> Hello,
> I have written a php script whose function is to detect dead links on
> a website.
> unfortunately,(obviously) my ISP  redirects invalid urls (or part of them) to a search site, so I get status 302 instead of 404.
> Even worse, in one case an invalid url (http://www.ajrccm.com)
> is also redirected to a search site (I don't know by whom) and I get
> the status 200 !
> Any idea how to solve this problem?

Change your ISP! Alternatively, my ISP also has this facility, but
offers a way to switch if off if you don't wish to use it.
Re: problem with link checker [message #179311 is a reply to message #179309] Wed, 03 October 2012 07:34 Go to previous messageGo to next message
M. Strobel is currently offline  M. Strobel
Messages: 386
Registered: December 2011
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
Am 03.10.2012 11:46, schrieb Helmut:
> Hello,
> I have written a php script whose function is to detect dead links on
> a website.
> unfortunately,(obviously) my ISP redirects invalid urls (or part of them) to a search site, so I get status 302 instead of 404.
> Even worse, in one case an invalid url (http://www.ajrccm.com)
> is also redirected to a search site (I don't know by whom) and I get
> the status 200 !
> Any idea how to solve this problem?
>
> thanx for your help, Helmut
>

1. Find a name server that does not spam DNS, see terminal log below.
2. on Linux I would not hesitate to install bind9, it's a one liner

/Str.
#
# host not found, okay
#
strobel@s114-intel:~> ping nonono.deee
ping: unknown host nonono.deee
#
# dig @ google nameserver, not found, okay
#
strobel@s114-intel:~> dig nonono.deee @8.8.8.8

; <<>> DiG 9.7.6-P3 <<>> nonono.deee @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 14380
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;nonono.deee. IN A

;; AUTHORITY SECTION:
.. 1800 IN SOA a.root-servers.net.
nstld.verisign-grs.com. 2012100300 1800 900 604800 86400

;; Query time: 92 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Wed Oct 3 13:27:20 2012
;; MSG SIZE rcvd: 104

#
# dig @ dnsadvantage, Spam!
#
strobel@s114-intel:~> dig nonono.deee @156.154.70.1

; <<>> DiG 9.7.6-P3 <<>> nonono.deee @156.154.70.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 644
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;nonono.deee. IN A

;; ANSWER SECTION:
nonono.deee. 600 IN A 92.242.144.2

;; Query time: 66 msec
;; SERVER: 156.154.70.1#53(156.154.70.1)
;; WHEN: Wed Oct 3 13:27:52 2012
;; MSG SIZE rcvd: 56

strobel@s114-intel:~>
Re: problem with link checker [message #179312 is a reply to message #179309] Wed, 03 October 2012 10:07 Go to previous messageGo to next message
Martin Leese is currently offline  Martin Leese
Messages: 23
Registered: June 2012
Karma: 0
Junior Member
add to buddy list
ignore all messages by this user
Helmut wrote:
> Hello,
> I have written a php script whose function is to detect dead links on
> a website.
> unfortunately,(obviously) my ISP redirects invalid urls (or part of them) to a search site, so I get status 302 instead of 404.
> Even worse, in one case an invalid url (http://www.ajrccm.com)
> is also redirected to a search site (I don't know by whom) and I get
> the status 200 !
> Any idea how to solve this problem?

Use http://validator.w3.org/checklink

This one is written in Perl.

--
Regards,
Martin Leese
E-mail: please(at)see(dot)Web(dot)for(dot)e-mail(dot)INVALID
Web: http://members.tripod.com/martin_leese/
Re: problem with link checker [message #179314 is a reply to message #179310] Wed, 03 October 2012 11:20 Go to previous messageGo to next message
Jerry Stuckle is currently offline  Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
On 10/3/2012 5:48 AM, Captain Paralytic wrote:
> On Oct 3, 10:46 am, Helmut <helmut_bl...@web.de> wrote:
>> Hello,
>> I have written a php script whose function is to detect dead links on
>> a website.
>> unfortunately,(obviously) my ISP redirects invalid urls (or part of them) to a search site, so I get status 302 instead of 404.
>> Even worse, in one case an invalid url (http://www.ajrccm.com)
>> is also redirected to a search site (I don't know by whom) and I get
>> the status 200 !
>> Any idea how to solve this problem?
>
> Change your ISP! Alternatively, my ISP also has this facility, but
> offers a way to switch if off if you don't wish to use it.
>

Hi, Paul,

Unfortunately, that isn't always possible - or prudent.

I have the same problem here, with no way to turn it off. There's only
one other ISP here, and we switched from them about 3 years ago because
of their poor service (internet was out for a week because of a "line
problem" - and not the first time!).

I hadn't thought about Matt's suggestion to change to a different DNS
because it hasn't been that much of a problem (I just do link checking
by SSHing into the server). But it's been a hassle at times (i.e. type
in a long URL but mistype one character - and have to enter it all over
again) that I'd like to cure.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex(at)attglobal(dot)net
==================
Re: problem with link checker [message #179315 is a reply to message #179311] Wed, 03 October 2012 11:21 Go to previous message
Jerry Stuckle is currently offline  Jerry Stuckle
Messages: 2598
Registered: September 2010
Karma: 0
Senior Member
add to buddy list
ignore all messages by this user
On 10/3/2012 7:34 AM, M. Strobel wrote:
> Am 03.10.2012 11:46, schrieb Helmut:
>> Hello,
>> I have written a php script whose function is to detect dead links on
>> a website.
>> unfortunately,(obviously) my ISP redirects invalid urls (or part of them) to a search site, so I get status 302 instead of 404.
>> Even worse, in one case an invalid url (http://www.ajrccm.com)
>> is also redirected to a search site (I don't know by whom) and I get
>> the status 200 !
>> Any idea how to solve this problem?
>>
>> thanx for your help, Helmut
>>
>
> 1. Find a name server that does not spam DNS, see terminal log below.
> 2. on Linux I would not hesitate to install bind9, it's a one liner
>
> /Str.

Hmmm, I have the same problem here but it's more of a hassle than a
problem (see my note to Paul). Hadn't thought about changing name
servers. Might have to do that. I've already got bind9 on my servers.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex(at)attglobal(dot)net
==================
Quick Reply
Formatting Tools:   
  Switch to threaded view of this topic Create a new topic
Previous Topic: button to send an email
Next Topic: Is Symfony 2 on-topic in this newsgroup?
Goto Forum:
  

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ]

Current Time: Mon Oct 23 22:29:46 EDT 2017

Total time taken to generate the page: 0.00719 seconds