Most Liked Content

Thought I'd write an update here!

So. It's been almost 6 years since I wrote the above notes. Most of it is still true; although I'm not sure I can really consider myself a newcomer to web development anymore!

I'm still the concertmaster of the Minnesota Philharmonic, and I'm still composing (albeit very slowly). In the web development arena, I still do accessibillity and usability focused development, and also am focused on improving accessibility within the WordPress ecosystem -- through themes, plug-ins, and core contributions. I'm the author of WordPress plug-ins with over a million downloads, so I spend a fair chunk of my time supporting those, as well...

I'm late to comment, but hopefully you'll excuse that this once.

Just wanted to add my own congratulations on the ... merger/aquisition/sponsorship?  Kim and Jim, Jim and Kim, both lovely people in my opinion, and both passionate about this business and environment.

Of the two, of course, only Jim has actually bought me booze, so there my loyalty has to lie.

So, what's new and upcoming next with Cre8asite and these other aquisitions?

Oh, and MarketPositionTalk forums were legend.  Susan Goodson was the wonderful moderator there, still remembered fondly.

Oh, and David Notestine (or something like that) was the guy with the Zeus 'robot', which was simply a tool that automated the building of reciprocal link directories, just as such were going out of fashion, and led to even greater speed in Google to kill them off.  To the point where David was peddling the software and half the people using it got a PR0 penalty for their reward.  No, I never, ever budge on people selling snake oil to the desperate, where it is likely to cause more harm than good.  (I still have an encyclopaedic memory of all things SEO )

So when the mighty Penguin struck way back in...I don't know when, but something like six months ago, a company came to me asking me to help them recover. Long story short, they needed to get rid of excessive footer backlinks, links that looked like paid backlinks (and some were), etc.

The really tough part? Getting the client to be patient and wait for another Penguin update to roll around so we could determine if the efforts were going to help or not.

Six months later. SIX MONTHS. To a client, six months of waiting is forever.

Client: "Should we do this? Will that help?"
Me: "No, nothing will help until Penguin runs again."

Client: "How about this? Would that help?"
Me: "No, nothing will help until Penguin runs again."

Six months of those questions and answers.

And now, FINALLY, Penguin has run again, and the client's rankings have recovered. Finally, I can get some peace. LOL!

Hello Cre8asiteForums Community!

It is my pleasure to announce that I have bought the Cre8asiteforums, and best yet, Kim Krause Berg will now be employed by Internet Marketing Ninjas.

Here is the official press release about Kim Joining Internet Marketing Ninjas, and the acquisition of the Cre8asite forums and blog.

There are many things that official press releases don't say, that I'd like to say to all of you.
To start with.....

YIPPYYYYYYY !!! I'm So Excited to be able to work with Kim and the Cre8asiteForums!!!
Thank you Kim.....thank you, thank you, thank you!

It's such an honor to be able to help Kim and the entire Cre8asite Community. This forum has been a large part of Kim's life work going back to 1998, and I know that there are so many others who have contributed countless hours towards making this community something really special. This is an opportunity I am humbled to have. The fact that Kim has entrusted me with the Cre8asiteforums....to say it's an honor is an understatement, it's an honor and a privilege....I'm excited...I'm a little scared. But my intentions are to, in a nutshell, "give you what you want to the best of my ability with the resources and time that I have available to me".  My main goal it to continue working to earn Kim's trust and to work towards earning the trust of the entire community as well. I am here to listen to you, not impose things on you. Keeping, and growing the community is the biggest value to me.
I ask that you give me a chance to show that I am here with the intention of making the community more valuable to you.  I believe that we can help each other...together, we can make the community something even more special than it is today....I believe in this....and I will need to rely on each member, each mod, and each admin, and Kim...because I haven't been in here much since I joined back in 2003. But, I am here to offer my resources behind what you want. I have many ideas, but in the end, it will come down to "What do you want?". I'll share my own ideas  with you soon...but I mostly want to hear what YOU want.. in the end, it is YOUR community.  Feel free to "Dream what you want" and we'll do our best to make that so.

I am extremely excited to have this opportunity to work with Kim, and with all the members of the Cre8asiteforums.  I have been given the chance to work with Kim, in both helping our clients to improve their sites, and to help the Cre8asiteforms become what all of you would like it to be. I have dreams of creating wonderful forums that are run by the community, and I'd like to ask all of you to dream with me, and then let's make those dreams happen.

I respect all of the forum members
I know that of all the communities I own, that the Cre8asiteForums is really special in it's own right. For many of you,  in many ways, it's a family. I am not seeking to change this, I want this to grow. Nothing is changing with the Admins or Mods, and I'll do what I can to keep and grow the community as you wish, so that there is even more value to all of you, the members.

I respect the Moderators, and Admins, and those have participated highly through out the years.
I have tons of respect for all you've done, and will work to support you. I know many of you have invested countless hours in giving back to the community, without your contributions the Cre8asiteForums would not exist. I know that you've been doing this, not for pay, but because you enjoy being part of this community. For all that all of you have done over the years, "Thank you". You've created something special here, and I'll work to preserve the good, and grow it, and improve it, with your input and help.

And most of all, I have the highest respect for Kim.
Kim is (and I image will always be) the main administrator. She built this community, and has maintained this community (of course, with all of your help)....and now, with your continued support, and with the help of some Ninjas, we can make this family something that sends positive energy around even more. I want everyone to be proud to be a member of the Cre8asiteForums...I will do all I can to work towards that goal.

Who is Jim Boykin?
Well, I'm almost always feeling lucky ... I'm the Founder and CEO of Internet Marketing Ninjas (originally was named "We Build Pages" from 1999-2011), which started as just me back in 1999, and has since grown to about 100 in house ninjas in Upstate New York. I have been lucky enough to surround myself with wonderful people who I highly respect. Last year we won a local Best Places to Work award (we hope to again this year), and recently several team members said some nice things about what it's like to be a Ninja.  In early 2012 I also bought the Developer Shed websites which included several several forums including another internet marketing forum called SEO Chat. I often speak at several conferences every year. I'm also a father, a boyfriend, a tree hugging hippy, and someone who tries to have a positive impact on the world....oh, and now I'm the owner of the Cre8asiteForums, and I'm going to strive to help make this community something even more special than it is already.

I love forums, and I love internet marketing, I grew up on SEO Chat and Webmasterworld, but I'm an internet marketing guy at heart...and have been since 1999. I love this industry more than you can image...and I believe in communities....and I see this wonderful community here. I see that for many of you, many of your fellow members are your friends...some even very close friends....there's a real family here...and I believe there's great power in communities, and I believe there's great power inside of the Cre8asiteForums community....and I'm here to lend my resources to helping to add value to the Cre8asiteForums.  I believe in Kim, and I believe in all of you, and I believe in giving back to my industry...this industry has given me so much, and if I can help areas of my community, and communities with my community (like the Cre8asiteForums), it all tends to come back in the end....call it Karma if you want...but I believe that if my impact on the Cre8asite community is good, then my business will benefit....I believe that if I can help all of you to benefit more from being a member here, then we all win...and I'd like us all to win.

I am asking for your help and support and faith in me. My intentions are nothing but good for the community. Together, I believe we can all get more benefit from our membership and participation in the Cre8asiteForums.

I'm Feeling Lucky,
Jim Boykin
Member Name webuildpages.
My Facebook,  Twitter, and Google

I have not only seen this done, I have done it myself! (granted it has been a few years since I last did it.)

 

What I used to do though was give away magazine subscriptions to owners of sites that were awesome. Buy the subscripts in bulk, and get them cheap each...can't beat a free yearly magazine subscription for one link.

The idea is that the more engagement you get from users, the more often your content is shown to users. By getting a lot of engagement on these silly FB posts, you stand a much better chance of getting your non-silly posts shown to users. It's a good strategy, and it really isn't harmful. Users like those kinds of things. It makes them feel smart when they can think of an answer when the post implies that they won't be able to.

 

ENGAGEMENT is an important buzzword; one that should be discussed more here, I think. I rarely post on my own blog anymore, but I just posted a few days ago for the first time in a long time, and it's related to the concept of engagement. It also shows a very very cool (in my opinion) way to engage users that I don't think many people realize is possible. It's one of those things you have to see to really "get" though, so you'll need to go read the post over there. Use this technique, and not only can you get more engagement, but more shares, more conversions, etc. Pretty powerful stuff. If you want to read the post, I've included it in my signature below. You should check it out. (Slightly off-topic, but not really...it's the kind of thing you could use on Facebook as well to get that same kind of engagement as the tactics mentioned above).

I would completely forget about absolute. There are very few cases where you need it and it's less flexible, which is increasingly important as screen sizes are varying.

 

Divs, paragraphs and heading are 'block' elements, by default they will stack one above the other in the order they appear in the code. Mostly you can just leave them to do that. So for example, to have a site with a header and foot, all you need is

 

.wrapper {width: 800px} - to set the width of the site

 

<body>

<div id="wrapper">

 

<div id="header">Header Text</head>

<div id="content">The content</div>

<div id="footer">The footer</div>

 

</div>

 

</body>

 

That's it no more divs or CSS. You can add spacing with margins.

 

You only need to tell the code where to put something if you want it different to that one above the other stack. Which is when floats come in handy - you want things to appear next to each other.

 

When you add float, instead of stacking one above the other, they'll move next to each other if there is space.

 

So inside your header div you set the two images to float and they'll got next to each other.

 

The same if you want to div next to each other..

 

<div id="header">

<div id="lefthead">Left text</div>

<div id="righthead">Right text</div>

</div>

 

CSS...

 

#lefthead,#righthead {float:left;width50%;}

 

Adding the width is important here, by default a div will be the full with of the page so they won't be any room next to it for the other div! So although they are floated they would still look one below the other if you didn't make them thinner.

Hello all - This is the author of GoDaddyHostingSucks.com.  No need to remain anonymous, my sites are actually listed at the bottom

Like the site states, I am one that does these activities many times a day and would hope some would at least listen my experience.

First, in a comment above one was challenging "Do we really expect these first line people to be really great?".  Well, in short yes.  In my experience with MANY MANY hosting companies, one that has blown me away with the level of support from front line people was HostGator.com  They know their stuff, and if the front line doesn't they push it upstream to some truly smart people.  I've have them help with technical issues on sites that I would have gladly paid hundreds of dollars for all done in the name of customer service.  With GoDaddy, it is 'refer to programmer'.

I don't claim to know 'why' this is, I just know it is.  Furthermore, I really don't know why or how HostGator can get away with that level of support and not charge.  I'm certain though it lends to their superior customer satisfaction rating.  At then end, with hosting costs a nominal as they are, why not pay for a company who knows what they are talking about?  Why would anyone pay roughly the same price and accept less when you don't have to?

Second, in the same post the poster also wrote "The person probably wrote this in a rage after the true story."  This is 100% true!   Actually the true story was the final straw to buy the domain name and put up the site, but yes - every experience I've had with GoDaddy's hosting has produced nothing but irritations, wasted time and naturally rage   What else would have produced such a site?

Finally, to circle back around to the original question of the forum topic "Who Is The Best Domain Registrar?"

... ironically I would say GODADDY.COM!

As mentioned on my site, their domain name registration services, DNS times, and interface I've always found to be very satisfactory, and actually recommend them frequently to my clients.  But their hosting is not.  So in short, buy your domains - as many as you want - from GoDaddy.  I feel they are a great registrar.  But go elsewhere with your hosting, you can get more for the same price if not less.

 

Hopefully this helps.  Thanks for your interest and comments.  Hope this message finds everyone well.

 

tucker

While I have a minor quibble or three (I can find a quibble anywhere ) with some of his points but Chuck Price absolutely hits an awesome zoomer in his article Unnatural Links Recovery: Emerging from a Manual Penalty, Search Engine Watch, 20-January-2013.

Disclaimer: I've not been adversely affected by either the B&W Fauna algo inputs nor received a manual penalty. I'm white as the snow I shovel at googlebot and currently even chillier...

However, if I ever want to get someone to remove a back link his template letter is pure extortion bliss:



Using uncomfortable truth to get desired reaction...
Note: I've long wondered how Google logs and reacts to such domain anti-G ToS behaviour...

I ran across How news organizations can sell sponsored content without lowering their standards by Jeff Sonderman, Pointer, 18-January-2013, just before I saw Donna's tweet...great minds bookmark the same sites

In summary it is about bringing magazine type advertising to the web - the news media portion that is - ala informercials, full page display ads, et al. The sentence that sums it up is Welcome to the brave new world of sponsored content.

The problem facing these sites is that they still have a serious problem 'getting it' - let alone getting it right - as was recently shown by the Atlantic's Scientology debacle covered by Poynter in Atlantic President: Our error was ‘in the execution’ of the Scientology campaign by Julie Moos, 19-January-2013.

I won't get into Google and other SEs normal attitude to ads, sponsored content et al because as we know such rules differ from small to medium to large enterprises, from vertical to industry to niche ad nauseum.

Further, although you may not know it, they are a decade late to the game and don't know the rules. How do I know? Because what they are currently fumbling is what I and others have been doing for that period as direct ads - but always behind robots.txt exclusions, with meta noindex for SEs that 'happen' to miss that no trespassing notice, and hard kick in the buttocks 403s when said bots are noticed where they aren't welcome.

Yup, all this and much more has been going on behind 'closed' doors for over a decade. The participating advertisers know of course but if they mentioned it to the big media organisations they forgot to mention the rules that make the game work. It is both sad and quite hilarious watching these lumbering giants drop the banana peels that they'll eventually slip on and fall... It's not rocket science, it's not even secret although it is quietly privately held information. Too bad they don't have people capable of basic research...

Similarly, the media moguls might want to buy a few lunches with some of the top individual affiliate webdevs...and stop re-inventing the wheel. Sad really, fortunately I have a fresh bowl of popcorn and the physical comedyand melodrama doesn't look even close to being over.

Way to go, Donna.  How will you find the time now that you're an award winning novelist too?
http://thegraveblogg...e-ribbon-award/
Many congrats.

Hi Everyone, My name is Joe Esposito, and I manage IT services here at Internet Marketing Ninjas.  Today we'll be doing the first phase of a two phase move.  This phase will incur the longest downtime and a period of read only access to the forums while we transfer the files and databases to our hosting infrastructure.  We anticipate it taking 2-3 hours from start to finish.  Figured I'd pop in and introduce myself before we begin today

I know we have been seeing a lot of movement but just around a handful of keywords over the past month, there have been players now showing first page than have come from no where (outside of top 50+), You should look at SEOmoz Google Algo update timeline to see if you can match any major movements in the past year to a drop or rise.  I suggest putting an annotation in Google Analytics for each key algo change or offsite change such as removing a sketchy link.  I know using tools like Cemper's LinkDetox tool have helped automated some of the backlink classification but you need to be sure that's the issue and not something else and manual inspection of your backlinks can often be the best method to be certain.

My example of what crushed me was the Exact Match Domain update really hit a few of my side projects hard and some went down by 65-95% in traffic but also the number of long tail keywords dried up.  Look at your Google Webmaster Tools and see if there is any shift in impressions/clicks as I know one site that was hit hard but on further inspection it was Google image traffic that had vanished.  If you have a country specific site I have seen a bulk of the international traffic you used to get has now dried up which can make it look far worse than it is.

But to your original question based on what we are seeing yes the damn bird is drunk and walking in circles and occasionally falls over with no consistent pattern and we have seen sites with spam/sitewide links escape getting penalised... so for some keywords/verticals we are at a loss on what Google is doing....

I have never advertised in the sense of buying ad space elsewhere on the web, however I do direct visitors via affiliate links and I do sell direct advertising, albeit mostly not a type of advertising most sites offer. So I have long had a serious interest in advertising, it's history and adaptations, and especially web advertising.

The basis for much advertising has moved from rational to persuasive. A comment of the late US Supreme Court Justice William O. Douglas illustrates this switch, "at the constitutional level where we work, 90% of any decision is emotional. The rational part of us supplies the reasons for supporting our predilections.". Emotion is visceral, a more fundamental brain activity than the rational. And most customers respond similarly, buying on emotion only subsequently marshalling facts to rationalise their decision.

So the recipe is quite simple if not actually easy: appeal to the emotional needs with sufficient supportive facts. The reason that it is not easy is (1) many/most advertisers make errors on both sides of the equation or defaults to silly, and (2) emotional needs are to some, often large, degree by one's society, place within it, and aspirations; thus making broad appeals that cross cultural, religious, ethnic, gender, economic, educational, generational, et al grounds difficult; one may attract some groups but repel others.

What sets a great writer apart is the controlled ability to suggest images and emotions by word choice and order, by the use of societal symbols, etc. This is as true of great copy as great literature. With the web there is an opportunity to personalise in both broad and narrow strokes, to filter audiences through their emotional responses down channels designed to both enhance specific emotional responses and proffer supporting rationalisations. Conversion funnels are a fascinating subject. To me at least.

Perhaps the most common mistake I see in ads and what I call ad copy (content designed as part of a sales process) is an emphasis on product/service features rather than customer benefits. Only developers care about all the features they sweated over, the customer only cares whether the features are a benefit, i.e. meet their need. Another error is not providing convincing proof of said benefits. Of course most ads, i.e. AdWords, are really a variation on the old print classifieds... so it becomes the ad linked landing page that is the real ad, which is why Google is so emphatic about landing page 'quality'.  
Note: No comment on their idea of quality.

Similarly with affiliate links... most affiliate links are just that, although some link after listing product/service features... such a waste. Not that I complain when competitors do this.

One of the hottest competitive ecom markets is jewelry. Think for a moment, not about the copy but the images. Many, if not most show the jewelry by itself. No sense of scale, of personalisation, of drama, of want and desire... Think of the impact of a glittering ring offered as if to you in an open box in a male hand... immediate emotion and symbol and... think of that ring on a female finger up as if you are admiring it... now add in appropriate captions and copy... There are many ways to transform products from mundane to magical, from here it is to here IT is, etc. If there is an emotional product it is jewelry and yet most of it is sold online exactly as a box of printer paper. Major fail.

And so it goes, industry by niche, so many ads, including ad copy (definition above), hitting all the wrong notes, leaving so many to wonder why online advertising is such a 'poor' performer, why so few affiliates score large. Just as so many have little understanding of SEO so too an horrendous lack of knowledge about advertising and how best to appeal to potential customers. Running a small business means becoming at least a jack-of-all-trades and that includes understanding advertising, sufficient, at least, to know who to hire and who not to produce it.

You need to separate your FB business and personal exposure.
I recommend building a separate business page and then modify business and personal privacy settings as appropriate.

Considering that my FB pages are all set to inactive you may want to take my input with a shaker full of salt

So, point by point.

1) Removing title attributes.

By default, WordPress adds title attributes to all system-generated links -- navigation menus, lists of posts, etc. These title attributes have exactly the same value as the text of the link -- if the text of the link is "Joe's Blue Widgets", then the title attribute is also "Joe's Blue Widgets". In the best case scenario, where a screen reader is set up to ignore title attributes, having this set makes no difference at all. In the worst case scenario, where a screen reader reads all attributes, it's as described by Kim above: it reads the title attribute and the link text, meaning that every link is, effectively, read twice. This is an enormous cognitive overload that benefits nobody. The purpose of the title attribute is to add context to a link that otherwise does not have sufficient context: but in almost all cases, the WordPress use does not meet that criteria.

2) Skiplinks.

Skiplinks are a means to help users of screenreaders jump from the beginning of a page to another section of the page. Visual users can scan a page quickly to identify the larger sections and skip over large blocks of text or links, but screen readers can't easily do that. Skiplinks provide a means to leap from the very top of a page to an alternate section of that page -- one principle use being to leap over the main navigation to the main page content.

3) Add site language and text direction to HTML element.

Screen readers use the site's declared language to figure out what pronunciation rules to use when reading the content. If no language is declared, then the screen reader will generally use the screen-reader user's installation's language or try to guess: e.g., if a French user of a screen reader visits your page, and no HTML language is declared, it will either read as if the text was written in French, or attempt to guess the actual language and pronunciation. Declaring the actual default language greatly improves comprehension of the text.

4) Remove target attribute from links

Links opened in new windows can break web site flow and disorient users who don't realize they've moved to a new window. Also, it takes away the option of opening in the same window/tab: all browsers have an option to open links in a new tab, few have an option to open in the same tab.

5) Force search error on empty search submission

The default WordPress behavior when a search is submitted with an empty search field is to simply return the main index with no notice at all. This is disorienting and confusing. The plug-in will change this so that the submission of an empty search field will return the same as a search with no results, which makes a great deal more sense in terms of user feedback.

6) Remove tabindex from focusable elements

Many (primarily older) themes had tabindex defined for comment input fields, in particular. Tabindex was an early concept for an accessibility feature for HTML, but was quickly demonstrated to be very problematic. What tabindex does is force keyboard navigation to follow a particular sequence according to the values defined in tabindex attributes. Removing those attributes usually makes keyboard navigation much more linear and predictable. The intention behind tabindex was to create a logical tab sequence when it would otherwise have been unnatural; but in practice, that required absolutely insane amounts of maintenance to use in a real-life web site. This option removes the attribute from standard focusable elements: links and form fields. It does *not* remove it from other fields, because a common technique to grant keyboard focusability to normally non-focusable elements is to define a negative tabindex: this places the element into the focus sequence but does not force it into a particular sequence.

7) Remove title attribute from inserted post images/featured images.

Similar to the title attribute above: WordPress automatically adds the title element to inserted images. This automatically strips it out; which is primarily a labor saver for people who would otherwise be doing this manually.

8) Add post title to "more" links

A common pattern for WordPress in many views is to show an excerpt of your post with a link to read the entire post. By default, this link is just plain text "more" or "continue reading" or an equivalent. This is problematic for accessibility because it means that there are many links on the same page with the same link text: for accessibility, all links should make sense without requiring context, to grant screen readers the ability to scan the page and have information. This plug-in adds the title of the post into this pattern, so that instead of hearing "Continue Reading. Continue Reading. Continue Reading.", the user will hear something like "Continue Reading About The Storm. Continue Reading The Storm Aftermath. Continue Reading Cleaning Up", and be able to pick the article they actually want to read from that set of links.

9) Add outline to elements on keyboard focus.

Many designs do not provide a design for the view of links or other focusable fields when a keyboard user brings focus to that element. This makes it very difficult for visual users who are keyboard dependent to keep track of where they are. A common pattern for this is to change the appearance of the element on keyboard focus; and this item adds an outline. Outline is rarely defined in most designs, so that ensures that the definition by this plug-in will almost always work and be visible; the same can't be said for text decorations like underline, since that may be the default state for links, and therefore would not be a change indicator.

10) Color Contrast Tester

This tool doesn't make any active changes to your web site, but allows you to easily test whether a set of colors you're considering for customization of your site meet the standards set by WCAG 2 for color contrast.

Hmmm. I should make this a blog post.

The October 2012 WebAIM (Web Accessibility In Mind) newsletter announces that our Moderator, Joe Dolson, released a new plug-in: WP Accessibility on 22-October-2012.

There’s only so much you can do via a plug-in when it comes to site accessibility — most of what grants accessibility for a WordPress site is in the theme. However, that doesn’t mean that you can’t do anything.
...
here’s what it can do so far:

* Remove redundant title attributes from page lists, category lists, and archive menus.
* Enable skip links with WebKit support by enqueuing JavaScript support for moving keyboard focus.
* Add skip links with user-defined targets.
* Add language and text direction attributes to your HTML attribute
* Remove the target attribute from links.
* Force a search page error when a search is made with an empty text string.
* Remove tabindex from elements that are focusable.
* Strip title attributes from images inserted into content.
* Add post titles to standard “read more” links.
...
At the moment, the plug-in is focused on front-end issues, and does not currently include any administrative improvements.

So, all you WordPress utilisers out there, go plug some accessibility into your sites...

Official WordPress 'WP Accessibility' Plug-in description and download page.

If you read my posts you know that while I value SEs, including Google, I have spent years diversifying traffic; that while I value AdSense, I have spent years diversifying revenue especially towards direct ad sales. I know that others have appreciated the value of such diversification but have either found it sufficient or simply easier to focus on Google and AdSense.

I blame webdev fora and conferences.

Why?
Because, until fairly recently, their emphasis has been on SEO/SEM. Primarily Google-centric.

While most webdevs were search focussed I was niche and marketing and advertising focussed. I attended or followed up on niche conventions, marketing and advertising conventions and symposia. While others networked within the webdev and search communities I networked within the niche and the mainstream marketing/advertising communities. While webdevs wanted access to Google engineers I wanted access to corporate marketing/advertising managers and CMOs, niche marketing/advertising agencies and their brand managers...

Do you see the mindset difference?

"Build for the visitor/customer" is a commonly heard mantra that I appreciate. Equally important is know your niche, the products/services available within, and their advertisers - but not filtered (solely) through AdWords or DoubleClick or other third party ad network.

The value of third party ad servers is, to the publisher: ease of use; to the advertiser: aggregation. For a solid niche authority once past 100,000 uniques (minus bots) a month it may be worthwhile considering going direct. Certainly once past 1,000,000. But to know who to approach, how, when, etc. you should have been in preparation: networking with agencies, with producers, et al. Very little of which happens (although it is starting) at webdev conferences. Change your midset, change your focus. And increasingly cut out the middleman.

I do note that this changes things: pajama development can remain but a suit and 20-minute presentations replaces jeans, teeshirt, and off the cuff comments... the emphasis shifts from web tech to web business - where it should have always been.

>  what are the possible causes?

The big thing a few people (such as Jim Boykins) were trumpeting at Pubcon was that (and I just got off a 14 hour flight and feel like I look so this is not entirely accurate) that one of Penguin and Panda (I forget which) was about links, and the other was about user signals like bounce rate.

One interesting comment was about pogosticking - which is where users go SERP -> click listing 1 -> click back to the SERP -> click listing 2 -> SERP -> listing 3 etc etc. When users don't find the answer they want, they keep looking.

The theory was - and no one I have as a client has suffered much so this is pure hearsay - was that if you don't provide users with what they want for a specific query, then you start to drop in position. As a really good example, if the first result for "do blind people dream" (https://www.google.c...nd+people+dream) leads people to come back to Google and click the second ranked page, that's a pretty strong signal that the 1st ranked page isn't solving the user's query.

FULL DISCLOSURE: I have no idea what will work, and I truly don't think anyone else does either, so I'd really be cautions about what you do to solve issues. That said, one exercise with minimal possible downside that you can do is check keywords with a high bounce rate in Analytics, and see if the page people get to actually solves that query. If not, look to beef the page up to improve the page's ability to serve the user for that query, e.g. if they get to a page with cheap flowers, make sure you have your CHEAPEST flowers on that page - not try to upsell. That is so very low risk an activity that I have no problem recommending, in fact it is a good idea fullstop.  Whether it actually helps solve your black animal issues or not, I truly can't vouch for.

Michael Martinez lets rip a good one in The Difference Between a Good Crawler and a Spammy Rogue Crawler, SEO Theory, 20-October-2012.

If you run a Web crawler you are a filthy spammer UNLESS you do the following:

* Publish a user-agent on your Website.
* Identify the user-agent in every fetch request the crawler makes.
* Honor the Robots Exclusion Standard (Cf. http://www.robotstxt.org/).

Your classification as filthy bottom-feeding scum for running a rogue crawler is not negotiable.

!!!!

So the Rules for Crawlers Are Simple

Rule No. 1: Respect the Website This means your crawler should identify itself and it should honor the Robots Exclusion Standard.

Rule No. 2: Make No Exceptions If in your bid to become the next big tech billionaire you find that a million Websites are blocking your user-agent and IP address, STAY OFF THOSE SITES. We don’t owe you anything. You owe US.

Rule No. 3: Don’t Take it Personally The Internet was not created so that you can make billions of dollars by stealing other people’s content and using it for your own benefit. So when you choose to do this and treat us as nothing more than ants to be crushed, we’re going to consider you to be bottom-feeding filth because that is what you are.

While I loved his rant and generally shouted Yes!!! or Huzzah! with every sentence I do disagree with this: ...Google honors the Robots Exclusion Standard...
Not always. And neither do Bing and other major SEs. They do run 'cloaked' bots, headless browsers, etc. as well as fudge on the 'meaning' of bot, user-agent et al to excuse apparent rogue crawler and fetcher behaviours.

It still behooves webdevs to build in appropriate bot defences in depth - unfortunately still no simple plug and play version available I know to recommend - but the parameters are only a search or three away, some even with basic code examples (caveat emptor). A typical website analysis shows 15-20% of it's traffic is bot; some exceed 80-90%. That is a lot of scraped content, wasted bandwidth, diluted conversion stats, etc. I even block most Google and Bing bots and other critters because they do not offer value in return; your needs may differ.

Great read. Thanks, Michael.