Jump to content

Cre8asiteforums Internet Marketing
and Conversion Web Design


Photo

Dramatic drop in number of pages indexed by MSN


  • Please log in to reply
35 replies to this topic

#1 bwelford

bwelford

    Peacekeeper Administrator

  • Site Administrators
  • 9019 posts

Posted 16 September 2005 - 06:27 AM

I get the feeling MSN search has dropped off most folk's radar screen. I don't know who may find this of interest but the number of pages indexed by MSN has dropped remarkably.

My website is remarkably stable, growing slowly with blog entries added at the rate of say 3 per week.

The number of pages indexed by Google and Yahoo! have behaved in a steady and predictable way. Currently Google shows 188 pages indexed and Yahoo! shows 238 pages indexed. 15 days ago MSN was showing 850 pages indexed and was presumably including individual Blogger posts as web pages. 3 days ago the number of web pages indexed dropped to 148. Today it's showing 48 web pages indexed.

Given the relative stability of my website, I think this is showing more about the behaviour of MSN. Has anyone else seen this drop in web pages indexed in MSN? Does anyone understand it?

#2 Ruud

Ruud

    Hall of Fame

  • Hall Of Fame
  • 4887 posts

Posted 26 September 2005 - 09:42 AM

Almost always when you see this kind of behavior on a search engine it is an update of some sort rolling out.

Right now I see 967 pages for you in MSN :)

#3 bwelford

bwelford

    Peacekeeper Administrator

  • Site Administrators
  • 9019 posts

Posted 26 September 2005 - 09:57 AM

Funny when I do a search for site:www.strategicmarketingmontreal.ca in MSN (via my neat FF extension Search Status 1.8), I still get 48 web pages indexed.

How are you doing your search, Ruud?

#4 Ruud

Ruud

    Hall of Fame

  • Hall Of Fame
  • 4887 posts

Posted 26 September 2005 - 10:03 AM

Same thing but on MSN itself.

#5 sebastienbillard

sebastienbillard

    Whirl Wind Member

  • Members
  • 95 posts

Posted 26 September 2005 - 10:17 AM

I noticed also some significative changes in the index, but not for all websites I monitor. Perhaps some index update ?

#6 bwelford

bwelford

    Peacekeeper Administrator

  • Site Administrators
  • 9019 posts

Posted 26 September 2005 - 10:57 AM

Curiouser and curiouser. When I click on that link in your post, Ruud, it shows what I'm always seeing - 48 results. :?

I suddenly had a brainwave. I was trying to think why the results might be different. My default browser is Firefox. So I logged into the C8 Forums using Internet Explorer. I then clicked on the link in your post, Ruud. I then got the result that there are 974 web pages. So MSN Search is detecting the browser used and serving up less pages if you use Firefox. Is there some sinister plot here?

#7 swainzy

swainzy

    Honored One Who Served Moderator Alumni

  • Hall Of Fame
  • 3322 posts

Posted 26 September 2005 - 11:14 AM

Barry,
I clicked on the link in the forum using Firefox and got the 974 links (?). :?
It seems to work for me.
D

#8 bwelford

bwelford

    Peacekeeper Administrator

  • Site Administrators
  • 9019 posts

Posted 26 September 2005 - 11:25 AM

Something very peculiar here, Donna.

I worked with both IE and FF side by side in windows and went directly to MSN Search in both. I get exactly the same result. Via FF it shows 48 and via IE it shows 974.

I then tried link: This time FF showed 49, while IE showed only 9.

Doing the same test with Google in both FF and IE, the numbers always correspond.

Help, what does it all mean.

Later addition:
One difference I note is that the pages that are displayed are different in the two browsers.

In Firefox, the SERP returned is:
http://search.msn.co...al.ca&FORM=QBRE

In Internet Explorer, the SERP returned is:
http://search.msn.co...al.ca&FORM=QBHP

However if I then take the address of each of the SERPs and use it in the other browser, this doesn't change the display. Firefox always shows 48 with either of these two addresses and Internet Explorer always shows 974 for either. :)

#9 Ruud

Ruud

    Hall of Fame

  • Hall Of Fame
  • 4887 posts

Posted 26 September 2005 - 11:46 AM

Barry, when you see the result page, press CTRL + F5. This forces a true refresh of the page vs. reading the page off your hard disk again. *Maybe* you're looking at a page which has been cached by your browser...

#10 AbleReach

AbleReach

    Peacekeeper Administrator

  • Site Administrators
  • 6467 posts

Posted 26 September 2005 - 11:50 AM

Later addition:
One difference I note is that the pages that are displayed are different in the two browsers.

So, if we're not checking SERPS with the most-used browser, we don't get the results that most users see??

Weird. I did not expect that at all. This'd be very counterproductive to competing browsers.

Elizabeth

#11 Ruud

Ruud

    Hall of Fame

  • Hall Of Fame
  • 4887 posts

Posted 26 September 2005 - 12:02 PM

IE, Opera, FF: 974 pages. I'm sorry Barry :) If you want I can try to contact them and get them to delist some? :)

#12 bwelford

bwelford

    Peacekeeper Administrator

  • Site Administrators
  • 9019 posts

Posted 26 September 2005 - 12:13 PM

No, there's still the mystery. Clearing the cache doesn't change the Firefox 48 count. I get 974 using Opera as well.

#13 Ruud

Ruud

    Hall of Fame

  • Hall Of Fame
  • 4887 posts

Posted 26 September 2005 - 12:24 PM

So what happens if you set Opera to identify as Mozilla?

My gut feeling tells me this is a local issue though. That or a data center - but you have this a long time now, right?

#14 bwelford

bwelford

    Peacekeeper Administrator

  • Site Administrators
  • 9019 posts

Posted 26 September 2005 - 12:37 PM

Opera returned the same results (974) whether identifying itself as IE, Mozilla or Opera.

I noticed a similar result for one of my customers with a TLD of .ca For that website, Firefox showed 24 web pages, while Internet Explorer showed 171 web pages. A few other .ca websites I looked at showed the same number either way.

#15 bwelford

bwelford

    Peacekeeper Administrator

  • Site Administrators
  • 9019 posts

Posted 26 September 2005 - 05:09 PM

Well the mystery seems to be solved. I guess MSN Search is broken.

The difference is explained by the fact that the MSN Search Settings for Firefox were at 50 results per page whereas those for Internet Explorer and Opera were at 10 results per page. If I change the results per page to 50 for MSN Search in both Internet Explorer and Opera, then these also show only 48 web pages indexed as compared with the 974 web pages if the setting is at 10 web pages per SERP page.

I've found this problem occured with at least half of the .ca websites I happened to check. How this can happen is beyond me.

It would be interesting to check whether others find the same change in results by changing the Settings value for number of results per page in MSN Search.

#16 swainzy

swainzy

    Honored One Who Served Moderator Alumni

  • Hall Of Fame
  • 3322 posts

Posted 26 September 2005 - 06:41 PM

Barry,

Where is the settings value? I have never noticed that choice. :?

Anyway, I just got 983 links for your site using MSN with FF and IE:
http://search.msn.co...SNH&srch_type=0

D

#17 Ruud

Ruud

    Hall of Fame

  • Hall Of Fame
  • 4887 posts

Posted 26 September 2005 - 06:59 PM

Yup. Settings 50 = 48 results. Settings 30 = 29 results. Settings 15/10 = 974 results. Offensive filter, grouping, location: none seem to influence it. I only see it with .ca sites, not with co.uk or fr. And within the .ca ones it isn't across the board either.

Weird.

#18 bwelford

bwelford

    Peacekeeper Administrator

  • Site Administrators
  • 9019 posts

Posted 26 September 2005 - 07:26 PM

Donna, the Settings link is just under the search box. .. and if I switch the Settings to 10 results per page, I get 983 web pages in Firefox too.

As Ruud says, definitely weird.

#19 BillSlawski

BillSlawski

    Honored One Who Served Moderator Alumni

  • Hall Of Fame
  • 15644 posts

Posted 26 September 2005 - 08:03 PM

Odder and odder.

I see the smaller amount for your site when I have MSN set to 50 results, and the larger amount when I have it set to 10 results.

So, I wondered how many it would actually show when I looked at all of the pages.

I did get this result on page 15, where it stopped:

Page 15 of 146 results containing site:www.strategicmarketingmontreal.ca (0.15 seconds)


But, I had to trick the search engine to get that far along. Following the pages sequentially, I got to page 8, and page 9 was the last one available. I backtracked to an earlier page, and it showed more pages at the bottom of the page. The furtherest along was page 15, where it showed the message above.

#20 swainzy

swainzy

    Honored One Who Served Moderator Alumni

  • Hall Of Fame
  • 3322 posts

Posted 26 September 2005 - 08:59 PM

Wow, I just don't see that at all. I hope I'm not embarassing myself:
http://www.msn.com/?...88f2ef86bbce1dd
:oops:
D

#21 bwelford

bwelford

    Peacekeeper Administrator

  • Site Administrators
  • 9019 posts

Posted 27 September 2005 - 04:09 AM

MSN is not like Google, Donna, which shows you its simple search page at its domain, www.google.com MSN is trying to intrigue you in to looking at lots of other things, since as we're seeing here it hasn't got the search thing right.

For MSN, you've got to go to http://search.msn.com

#22 swainzy

swainzy

    Honored One Who Served Moderator Alumni

  • Hall Of Fame
  • 3322 posts

Posted 27 September 2005 - 01:17 PM

Barry,
I see it now. Thanks for your time and patience. :lol:

I do believe you are right:

MSN is trying to intrigue you in to looking at lots of other things


Thanks again,
D

#23 A.N.Onym

A.N.Onym

    Honored One Who Served Moderator Alumni

  • Invited Users For Labs
  • 4003 posts

Posted 19 May 2006 - 04:27 AM

Though not entirely dramatic, I have experienced some random index dropping from MSN.

I have submitted my site to about 100 directories (SEO friendly), and about 25 of them have set up links for now and got indexed. Within a couple of weeks since submitting, four pages got indexed.

Then funny stuff began happening. A week ago there were 3 pages shown in MSN.
Today there is onle one page in MSN.

Google and Yahoo have been slow on including in the index and only show the homepage so far.

#24 Mano70

Mano70

    Mach 1 Member

  • Members
  • 256 posts

Posted 19 May 2006 - 07:39 AM

MSN is a joke for the moment. I have also seen pages dropping from several domains, but for one brand new domain with no more than 15 pages MSN claimes that it has indexed/are showing 63 pages. (But when you are paging page 3 to 6 in the search are identical).

#25 A.N.Onym

A.N.Onym

    Honored One Who Served Moderator Alumni

  • Invited Users For Labs
  • 4003 posts

Posted 19 May 2006 - 08:10 AM

Sad to hear that.
I was intending to get my deserved 20% share of total search engine traffic from MSN.

Another fact I remembered.

First it was 4 pages. Then it went to 3 pages. Then back to 4.
This is pretty weird, considering the boat has visited all the pages.
Anyway, we had some issues with being available (during the night) and now MSN index reflects that.
Oh well.

Well, I am working to provide value to my visitors, with or without MSN, so it doesn't matter much.

#26 yannis

yannis

    Sonic Boom Member

  • 1000 Post Club
  • 1634 posts

Posted 23 May 2006 - 10:59 AM

These symptoms are very similar to Googleitis see Thread. I think MSN just caught it from Googlebot!

Sorry everyone I just could not resist it!!!

#27 greendude

greendude

    Ready To Fly Member

  • Members
  • 10 posts

Posted 27 May 2006 - 12:35 PM

I find MSN indexes perfectly if you follow all the known basic rules for SEO. Good short descriptive Title, Proper keywords for the content of that "page", Desc to match page content, Re-index as per meta tags etc etc.

By following the rules of what Google recommends for best SEO, I have been coming up in MSN in the first page of search results for over 100 top keywords (sometimes 2nd but seldom third - a few keywords have to much competition and i am farther down the list - not often though) - and every single page from my site is indexed - including every possible link to my site - MSN shows over 470 links to my site and I was only aware of about 40 - so they have found many that may be several from the same site - in other words, MSN seems to have found every possible link and page form my site and indexed it. I am not sure how a DMOZ listing helps as I was listed on DMOZ within a week but have not seen any benefits on Google or Yahoo.

So I don't feel MSN is lacking in the indexing of sites. Not only am I well indexed, I was on MSN within 2 weeks of submitting the site and in the top results pages within 4 weeks.

I think site content and complete pages without broken links etc all help a lot.

Ironically - I followed recommendations from Google for SEO (as well as many sites that seem good - and this forum) only to be listed very well in their competition's Search Engine MSN, but nothing for Google or Yahoo

Go Figure??

By the way, I personally do not under estimate MSN. Microsoft is going all out with their adsense program - I tried it with very good results in more than one way - the following show me Microsoft is serious.
  • Low cost and very good CTR
  • Outstanding Customer support - never more than two rings and you are talking to a human and they have no prob talking and working aout a problem with ads or related items - even if it takes 30 or more minutes - which costs them more than I was paying
  • More stable ad costs and good display position of ads compared to cost
  • Unlike Google Adwords if you bid low on MSN you still appear, maybe farther down the results pages, but there all the same - google drops you if you don't bid a certian amount - not MSN
  • Overall my cost per keyword averaged .04 on MSN and always show somewhere. .11 cents on Google and conastantly have to check the keyword to see if it is active (increase bid to stay active) not so on MSN, if you are outbid you go down the list but still appear at some point.
Overall, I would would pick MSN over Google for cost, top service and results - I had both Adword programs running at the same time and 70% came from MSN, 30% from Google with Google costing almost the same as my MSN campaign.

Anyways, the point I am trying to make is that I believe MSN is serious about capturing their share of the search market and ad revenue, and we all know that when Microsoft wants something, they usually get it - look at the browser most of us use today. Firefox has made a little dent, but my site stats show 95% of visitors using IE.

So far MSN seems to index exactly as per recommended SEO. In fact, of all the SE's, I find MSN rewards you if you do things by the book. Yahoo and Google ??? It is no wonder website owners try all sorts of tricks to get listed on Google or Yahoo - if you follow their own recommendations, you get didly squat for the effort - at least MSN looks at your site, says - yup, followed the rules, in ya go - followed the rules and lots of good content - well, MSN will bump you up the list.

Very confusing to follow the rules as suggested by Google (including sitemap.xml file etc) and get no results in return - even when you can see the last time they indexed your site - so you know they have been there.

Even after stopping my Ad campaign with MSN (to see if that had an effect on my listing) my placement in results has not changed. I see MSN as a very good source of traffic. I get lots of traffic from MSN and excellent CTR on my ads, anywhere from 5% to 30% with an overall average of 7% so the traffic can be deemed quality traffic.

Edited by greendude, 27 May 2006 - 01:07 PM.


#28 Gr8ted3s

Gr8ted3s

    Gravity Master Member

  • Members
  • 152 posts

Posted 30 May 2006 - 10:11 PM

Well hello everyone! Remember me?

I haven't been cheating on cre8asite....I've just been REALLLLLLY busy. I still owe sanity a reply to a message dated 12/6/05! :) Sorry Sanity, I'll make it up to you somehow :cheers:

Now that I'm back, I haven't had a chance to see what everyone is talking about. One thing I've noticed (or noticed not...which is just as strange) is that nobody is talking about MSN.com using live.com data...what am I missing here? Can't be old news already can it?

AND alexa.com appears to be using live.com data as well, sometime within the last two weeks. Figures, I step out for a couple of weeks and the canvas gets a new painter!

At any rate.....whatchya all been doing for the last six months!?!?!?

#29 BillSlawski

BillSlawski

    Honored One Who Served Moderator Alumni

  • Hall Of Fame
  • 15644 posts

Posted 30 May 2006 - 10:55 PM

Nice to see you, Gr8ted3s

MSN tends to be overlooked here some, and from my visitor logs, by searchers too.

I like some of the papers that I've been seeing come out from the folks at Microsoft, and I'm actually seeing them index sites that I'm watching fairly well. As Barry Schwartz noted on the SEW blog, the press sort of ignored the changes you mention, too:

http://blog.searchen...g/060501-084415

It's been an interesting year so far. Hope that you've been busy in a good way, and have some time now to hang around with us.

#30 bwelford

bwelford

    Peacekeeper Administrator

  • Site Administrators
  • 9019 posts

Posted 31 May 2006 - 04:29 AM

As Bill said, there's very little traffic comes my way from MSN Search so that's one strike against it. More worryingly I find that searches such as site:www.mydomain.com and link:www.mydomain.com produce results that vary enormously from time to time. I guess they're still working on it so I don't bother with it until it is more stable.

I realize that there are whole slices of the population out there who are into different things from me. That Myspace crowd, I'm told, is pretty big, but very little comes onto my radar screen.

Now suddenly this morning I'm presented with another one. Where did live.com sneak up from? I'd completely missed that one. Took a quick peak and still not sure what it's all about. Tried the live.com blog but it broke in Firefox so didn't read too much. Please can anyone tell me in a few words what live.com is trying to do and how this relates to other MSN products. How does it relate to other competitive products?

#31 A.N.Onym

A.N.Onym

    Honored One Who Served Moderator Alumni

  • Invited Users For Labs
  • 4003 posts

Posted 31 May 2006 - 05:00 AM

Live.com doesn't work in Opera as well.
I guess MS still lives in its own little world of IE.

It seems like Live.com is just another product to promote search from MS.
At least I judge by the amount of search options it has.

As to how it relates to other competition, I don't see anything obvious here.

#32 Guest_joedolson_*

Guest_joedolson_*
  • Guests

Posted 31 May 2006 - 08:40 AM

It's quite annoying how frequently Microsoft creates IE only resources - it's like they simply WANT to alienate the tech crowd.

Regardless, the "Live" brand is a big branding mess...there are some interesting notes on it at ZDnet - Microsoft's Brand Confusion (from Feb) and Microsoft admits brand confusion from April...

Supposedly, "Live" is now Microsoft's brand for services and "MSN" is their brand for content. But the whole thing just makes for a big mess...

#33 bwelford

bwelford

    Peacekeeper Administrator

  • Site Administrators
  • 9019 posts

Posted 31 May 2006 - 09:23 AM

You only know that you have a brand, when your customers show you by their actions that the brand is an important concept to them. You can't have two brands IMHO.

I think Microsoft should go back to the drawing board and decide what they're all about. Who knows perhaps Microsoft should be the brand. It might just work. :)

#34 Gr8ted3s

Gr8ted3s

    Gravity Master Member

  • Members
  • 152 posts

Posted 31 May 2006 - 11:13 AM

I think joedolson says it pretty well. Live.com is their Web 2.0 product. I can't get over that...Web 2.0 hahahahaha.

They seem to be using live.com for many things, two of which are entering the product comparison market from different directions. They're also doing something funny with "gadgets"....mini programs, or micro software...sorry, couldn't resist. Kinda sounds familiar.

As far as p***ing of the tech world, I don't know. They seem to be moving toward a dev friendly back end. Don't underestimate MS. People have done that in the past. MS may have a long way to go, but they have a way of sneaking up on you.

The thing that surprised me was that they would start using the results in live.com in msn.com and alexa already. Seems a bit premature.

Bill -

Good, bad, it's all relative anyway. Mostly good, some wasting of time, which of course can be fun unless you are forced to waste it in ways you don't want to, which seems to happen from time to time. Yeah, I'll likely be around a bit. I tend to get stuck on projects though and when that happens I'm prone to disappear for months at a time....as you just witnessed.

Thanks for the link too, I still try to get over there from time to time as well, but there's usually so much stuff to wade through. All in all they do a pretty good job of sorting it all out though. Good to be back.

Have many of the flock that I came here with stuck around?

#35 BillSlawski

BillSlawski

    Honored One Who Served Moderator Alumni

  • Hall Of Fame
  • 15644 posts

Posted 31 May 2006 - 11:49 AM

Have many of the flock that I came here with stuck around?


There are a few who hang aorund - some more than others. SEO Refugee is a nice community that many from that group have also been involved with, and others have been active at the Search Engine Roundtable Forum, though all are welcome here, too.

I hear you on the projects. I spent about 20 hours over the three day weekend to tweak a site that I expect will now cost me an hour or so a day. Good thing I'm enjoying it.

I wonder if Microsoft's two different approaches to shopping confuses them, too.

The idea of Microsoft Live as a customizable portal isn't a terrible one, and Google's doing the very same thing with their personalized search. And even Yahoo!, portal that it is, has a clean looking search interface at http://search.yahoo.com/

#36 IrvPWC

IrvPWC

    New To Community

  • Members
  • 1 posts

Posted 02 May 2007 - 09:29 AM

Between 3/21/07 and 3/26/07 I noticed a big drop in "# of pages indexed" and "external links into site" by MSN on two different sites that I monitor.

The original "# of pages indexed" looked quite high for one site, so I did some research and found that it was counting each file (htm, html, jpg, gif, css, swf, etc.) as a page. I figured that MSN finally fixed that problem to give more reasonable results, and therefore my external links went down also.

Has anyone else noticed this?



RSS Feed

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users