Jump to content

Cre8asiteforums Internet Marketing
and Conversion Web Design


Photo

Your First Seo Tool


  • Please log in to reply
14 replies to this topic

#1 eKstreme

eKstreme

    Hall of Fame

  • 1000 Post Club
  • 3399 posts

Posted 16 December 2006 - 06:33 AM

So as you guys were waiting for me to write my 1000th post, I got a very badly timed case of... writer's block! Apologies for the delay! Inspiration has now arrived, so here goes: How to write your first SEO tool.

We will use the Yahoo! API because it is the easiest to learn and use, and their dataset is huge. Also, the have a funky operator called linkdomain:. So our tool will do something very simple: you feed it a domain name, and it searches Yahoo, using the API:
linkdomain:mydomain.com -site:mydomain.com

Also, we'll be using PHP5. It makes life so much easier!

Let's roll.

Yahoo! Developer Network and API 101
The most important site you need to know about in this tutorial is the Yahoo! Developer Network. There you can find code samples, documentation, mailing lists, and everything you need to get going for all of Yahoo's APIs. Read it, love it, learn it, and most certainly bookmark it!

The Yahoo API is done over a protocol called REST. Don't worry to much what that means, but what we care about is its simplicity. With REST, you construct a URL that contains instructions (our 'command' for the API) and then request that URL from the server, as if you're browsing. Instead of seeing an HTML webpage, you get valid XML.

So our task is two-fold:
  • Construct the URL and request it (as if we're browsing for it)
  • Retrieve and parse the XML
Once we have the XML, we can save it to a database, print it out on screen, or whatever. Our example will just print out an orderly list on screen.

The Yahoo API requires that each application using it identify itself. The identification is straightforward: you sign in to the YDN and register a unique Application ID. The demo ID that Yahoo uses for all its examples is YahooDemo, so we'll use that here. You need to get your own if you're going to be using the API extensively.

Another thing: the API is limited to 5000 queries a day per IP address. That should be fine for everyone, but if you get throttled suddenly, this might be it :)

Setting Up
First we create the input form and the basic structure of the page:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Linkdomain Search</title>
</head>
<body>
<h1>Lindomain Search</h1>
<form action="<?php echo $_SERVER['PHP_SELF'];?>" method="get>
Domain: <input type="text" name="thedomain" /> <input type="submit" value="Find Links" />
</form>
</body></html>
Nothing exciting: just a simple XHTML page with a form. The form submits to itself a variable called thedomain. You can save this code to any file name you want.

Next up, we add some logic to see if we have a domain name to process. If we do, we retrieve the domain name and sanitize it a bit (for security).

if(isset($_GET['thedomain'])){
$domain = strip_tags(trim($_GET['thedomain']));
}
Next we can construct the search, which is:
$search = urlencode("linkdomain:$domain -site:$domain");
Notice that we URL-encoded the search because we're starting to construct the REST URL now. So a quick recap at where we are, the full code is:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Linkdomain Search</title>
</head>
<body>
<h1>Lindomain Search</h1>
if(isset($_GET['thedomain'])){
$domain = strip_tags(trim($_GET['thedomain']));
$search = urlencode("linkdomain:$domain -site:$domain");
}
<form action="<?php echo $_SERVER['PHP_SELF'];?>" method="get>
Domain: <input type="text" name="thedomain" /> <input type="submit" value="Find Links" />
</form>
</body></html>

Constructing the REST URL
Now we fully construct the REST URL. The base for all Yahoo API web search service is http://search.yahooa...ce/V1/webSearch . To this, we append our query and some other bits and it's ready:
$SearchURL = "http://search.yahooapis.com/WebSearchService/V1/webSearch?query=$search&start=0&results=50&appid=YahooDemo";
The query string variables are easy to explain: we have $query which is our... er... query. We want the results that start at 0 (programmers are funny that way because we sometimes start counting at zero instead of one), and we want 50 results. Finally, we append our application ID and we're set.

Retrieving the XML
Now that we have a URL, we need to 'browse' to it. In reality, we'll actually fetch the URL and what we fetch will be the XML.

Now here is a trick: APIs can be unstable, especially Google's. Sometimes, if you query an API, it returns an error. A second later, with the exact same query, it works. Of course, we can put in error management, but wouldn't it be nice if our program tried more than once on our behalf? Of course it would be.

To accomplish this, we create a loop that says "keep trying this until you get some XML or you've tried 3 times and wait a second between tries". In PHP:

do{
	$xml = @simplexml_load_file($SearchURL);
	if(!$xml){
		$Tries++;
		sleep(1);
		}//eob if(!$xml)
	}while(!$xml && $Tries < 4);
Now we check if we actually got some XML, and if we did, find out how many results were returned:
if($xml){
foreach($xml->attributes() as $name=>$attr) $res[$name]=$attr;
$results = $res['totalResultsReturned'];
echo "<p>Found $results result(s)</p>";
}
Now we continue: we parse the XML (technically 'traverse the XML') and display the results:
for($i=0; $i<$results; $i++) {
	  foreach($xml->Result[$i] as $key=>$value) {
		  if($key == "Url"){
			$url = $value;
			  }
		  if($key == 'Title'){
			 $title = $value; 
			  }
		  if($key == 'Summary'){
			 $snip = $value; 
			  }
		 }//eob foreach
	echo "<div class=\"resultdiv\"><a href=\"$url\" target=\"_blank\">$title</a></div>";
	echo "<div class=\"snipdiv\">$snip</div>";
	  }//eob for loop
	}
Finally, remember we checked using if($xml). What happens if even after 3 tries we still didn't get any XML? We have to display some form of error. With this final addition, we're done! The full code is:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Linkdomain Search</title>
</head>
<body>
<h1>Lindomain Search</h1>
<?php
if(isset($_GET['thedomain'])){
$domain = strip_tags(trim($_GET['thedomain']));
$search = urlencode("linkdomain:$domain -site:$domain");
$SearchURL = "http://search.yahooapis.com/WebSearchService/V1/webSearch?query=$search&start=0&results=50&appid=YahooDemo";
do{
	$xml = @simplexml_load_file($SearchURL);
	if(!$xml){
		$Tries++;
		sleep(1);
		}//eob if(!$xml)
	}while(!$xml && $Tries < 4);
if($xml){
foreach($xml->attributes() as $name=>$attr) $res[$name]=$attr;
$results = $res['totalResultsReturned'];
echo "<p>Found $results result(s)</p>";
for($i=0; $i<$results; $i++) {
	  foreach($xml->Result[$i] as $key=>$value) {
		  if($key == "Url"){
			$url = $value;
			  }
		  if($key == 'Title'){
			 $title = $value; 
			  }
		  if($key == 'Summary'){
			 $snip = $value; 
			  }
		 }//eob foreach
			echo "<div class=\"resultdiv\"><a href=\"$url\" target=\"_blank\">$title</a></div>";
			echo "<div class=\"snipdiv\">$snip</div>";
	  }//eob for loop
	}
else{
echo "<p>No XML was retrieved :(</p>";
}
}

?>
<form action="<?php echo $_SERVER['PHP_SELF'];?>" method="get">
Domain: <input type="text" name="thedomain" /> <input type="submit" value="Find Links" />
</form>
</body></html>

What does it look like? Check it out!

Improvements
For one thing, we didn't go into any error management - just basic stuff. Also, the tool is specific for linkdomain searches which we can improve. We're using just the Yahoo API (which in this case makes sense) but we can add Google and MSN APIs too.

Randoms
Now I want to answer some questions from this thread:

What are your future plans?

World domination... with a lot of sleep. Seriously though: I'm working on a very cool project that I'll be needing some testing help and linklove for in a couple of weeks :)

More obscure trivia about your site(s) - I'm sure you have some interesting stories to tell!

I've had people from IBM, Microsoft, Google, NASA, the US Airforce, US Military, Philips, and many more visit my sites.

Funniest tech support story: a woman in an Arizona Uni emailed me for help with one of my scripts. Why? Because she was the new boss and didn't want to look weak in front of her programmer employees :D

chaos theory applied to SEO

PageRank! It's a measure of the probability that a random surfer will land on the page.

I always want to know what kind of thing makes a person's day.

Web-wise: a nice email from a visitor or a funky new link. Generally: the satisfaction of an achievement, be it learning something new (I remember the first time I figure out a perl guestbook), or figuring out something obscure, or finding a solution to some problem.

Is there anything about being here that keeps you jazzed?

The knowledgeable people. I didn't participate in so many threads simply because I had nothing new to add! It's like brainwave central!

Shoutouts and thanks: Kim, Bill, John, Elizabeth, Joe, Yuri, Ruud, Rand x 2, Barry, Wit, and many many more I'm sure the weekend hangover is making me forget!

Pierre

#2 A.N.Onym

A.N.Onym

    Honored One Who Served Moderator Alumni

  • Invited Users For Labs
  • 4003 posts

Posted 16 December 2006 - 06:58 AM

Yay, finally!

Thanks for the useful tool creation instruction.

Be careful, I'll learn how to make tools and take over the world, too!

Thanks again :D

#3 JohnMu

JohnMu

    Honored One Who Served Moderator Alumni

  • Hall Of Fame
  • 3518 posts

Posted 16 December 2006 - 07:00 AM

Thank you, Pierre! Not just for 1000 great posts here but the millions of ideas that are triggered with your tools, websites and posts everywhere! :flowers: :flowers:

Great idea for a 1000th post!

John

PS one more question: where did the name "eKstreme" come from?

#4 bwelford

bwelford

    Peacekeeper Administrator

  • Site Administrators
  • 9008 posts

Posted 16 December 2006 - 09:19 AM

Congratulations, Pierre. That's quite a challenge you've created for anyone else who wants to do a very special 1000th post.
:applause:

#5 eKstreme

eKstreme

    Hall of Fame

  • 1000 Post Club
  • 3399 posts

Posted 16 December 2006 - 09:19 AM

Glad you like it guys! Yuri, if you can write tools, that would be great. Competition drives innovation, and we so need new ideas in SEO tools!

PS one more question: where did the name "eKstreme" come from?

Oh, good question. Way back in 1999, I heard of a company that offered free domain names. It was called NameZero and it registered domains for free in exchange for a frame with an advert. I was still hesitant to pay for a domain name because I was still not sure if my new hobby was serious or not, and so this opportunity was a great solution.

So I started thinking of names. The usual "pierrefar.com" and the like came up but I wasn't excited enough. One day, I was procrastinating and found a website about "extreme ironing". For the uninitiated, that involves (I'm not kidding) take an iron, an ironing board, and a shirt, hiking up to the top of a mountain with them, and ironing said shirt at the peak. Amused, this whacky idea stuck, and so I tried to register extreme.com. Back then, that was the official website of the Extreme Sports association or something. Bummed out, I went to bed and then it hit: back then, typo domains were the latest fad, so the misspelling of extreme is... ekstreme! I got out of bed and sure enough it was free. Success!

This created an amusing situation: ekstreme is the Swedish spelling of the extreme in English. So once in a while, my website search gets a few odd searches... about ummm, the birds and bees :D I really, honestly do not want to know more!

And that's it!

MORE THANKS: I can't believe I forgot these (many apologies!) but great thanks to Amon, Garrick, swainzy too. I'm sure I'm forgetting more, so apologies in advance!

Pierre

Edited by eKstreme, 16 December 2006 - 09:21 AM.


#6 Guest_joedolson_*

Guest_joedolson_*
  • Guests

Posted 16 December 2006 - 11:08 AM

Awesome 1000th, Pierre!

Using the various API's available is something I've just never gotten around to delving into...now, though, it may become so much easier! Mind if I help you with your world domination? I've always wanted to be an evil henchman.

Thanks!
-Joe

#7 Wit

Wit

    Sonic Boom Member

  • 1000 Post Club
  • 1599 posts

Posted 16 December 2006 - 11:34 AM

Dude, that is a very cool 1000th. Gives us all something to play with over the weekend (and the holidays LOL).

As a matter of fact, I've come to a point where I need to use a lot of that stuff for my own projects. Consider your code nicked hehe.

PS: good thing you didn't wait for your birthday to post, but then again I dunno when your birthday IS. :D

#8 bragadocchio

bragadocchio

    Honored One Who Served Moderator Alumni

  • Hall Of Fame
  • 15634 posts

Posted 16 December 2006 - 03:03 PM

Great stuff, Pierre.

Thank you.

I've been meaning to start out messing with APIs with the Google Maps API, because I like the visual aspect of it. But this is a great introduction.

Congratulations on the 1,000th post, and thanks for making it such a terrific post.

#9 earlpearl

earlpearl

    Hall of Fame

  • Hall Of Fame
  • 1599 posts

Posted 16 December 2006 - 05:23 PM

That is soooooooooooooo great pierre.

Congrats on 1,000 posts. :applause:

Dave

#10 jadibones

jadibones

    Ready To Fly Member

  • Members
  • 15 posts

Posted 16 December 2006 - 09:19 PM

Congrats on the 1000th. Don't go and give away all the secrets. :D

#11 Cath

Cath

    Mach 1 Member

  • Members
  • 370 posts

Posted 16 December 2006 - 10:00 PM

Congrats Pierre :applause:

That's a cool tool, I've got backlinks I never knew I had :D

These 1000 posts just get better and better, the way things are going pretty soon someone will be posting the secret of the Philosopher's Stone, or perhaps the elixir of youth :)

#12 cre8pc

cre8pc

    Dream Catcher Forums Founder

  • Admin - Top Level
  • 13527 posts

Posted 16 December 2006 - 10:07 PM

Congratulations Pierre!

We are the better here for your generous spirit and creativity.

For more of his SEO Tools, check out our Tools Page. We've been fans here for awhile :D

#13 rmccarley

rmccarley

    Light Speed Member

  • Members
  • 642 posts

Posted 17 December 2006 - 02:58 AM

Thanks eKs. I needed that. :D

And congrats. getting 1k posts in with this brain trust is a real accomplishment.

#14 eKstreme

eKstreme

    Hall of Fame

  • 1000 Post Club
  • 3399 posts

Posted 17 December 2006 - 06:08 PM

Thanks for your kind words everyone. Your generosity and help are what make me want to contribute here. It's been true ever since my very first post, and it just keeps getting better.

I'm very happy you guys liked the code and I'm glad a few people are saying they're going to give APIs a shot. That's great! Please post back here with your results and questions :) BTW: Do you think there is enough demand to start an SEO Tools forum here? Just thinking out loud here...

Mind if I help you with your world domination? I've always wanted to be an evil henchman.

Welcome aboard Captain Joe. Please report to the tailors so we can custom-make your evil looking, but very usable, suit.

PS: good thing you didn't wait for your birthday to post, but then again I dunno when your birthday IS. :)

End of October so it would have been a while. Hope the code serves you well.

I've been meaning to start out messing with APIs with the Google Maps API, because I like the visual aspect of it. But this is a great introduction.

The maps API is more Javascript based while we're talking PHP-based APIs here. I do have a toy project I play with once in a while using the G Maps API. Maybe I should brush it up and publish it. Too many ideas, too little time!

These 1000 posts just get better and better, the way things are going pretty soon someone will be posting the secret of the Philosopher's Stone, or perhaps the elixir of youth :)

Well now that you've mentioned it, unless someone beats me to it, I thought that would be my 2000th post :)

So to reiterate: please post back your tools and questions here so we can all admire and answer you :)

Pierre

#15 Cath

Cath

    Mach 1 Member

  • Members
  • 370 posts

Posted 21 December 2006 - 07:58 AM

Here's one I made earlier :D said in my best Blue Peter accent

With some help, and changing my htaccess so it would use php5 scripts, I now have my first seo tool :)

So, do I get my Blue Peter badge now?

Thanks Pierre :applause:

Edited by Cath, 21 December 2006 - 08:06 AM.




RSS Feed

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users