Did you redirect all old pages to the relevant new ones?
That's the key factor right there. If you used to have a page called site.com/about.htm you either need to have a new page with the same name, OR you need to redirect all of those old pages so that site.com/about.html redirects to site.com/about/ (there are several ways to do this - including a plugin for wordpress that lets you redirect stuff to WP pages/posts). Granted, if this has been going on for a while now (i.e. it's been more than a month or so) this won't help with search ranking, but it could help with people who have pages bookmarked on the old site.
My best advice would be to NOT go back. Going back, especially if you use the same strategy and you did going forward, will only reproduce the same results. It's not WP itself that is causing the drop - it's the new URL's, the new way the site is arranged, and all of that. Chances are, because it's wordpress, there is a lot more "replicated elements" on each page in widgets and so on. Even though WP has a generally standard page structure, Google especially doesn't like to assume that the content is where it is on most WP sites - it needs to take some time and try to figure it out.
Another reason not to go back is that over this past month or whatever, Google has forgotten your old site and is trying to learn your new site. Going back makes it throw away the month (or whatever) of learning it has already done and start again from scratch. It's NOT going to just say, "Oh, the old one is back, let me rank it the way I used to."
The merging of the sites shouldn't cause any long-term issues, but because of the above, it will cause short term ones. It is basically a new site and so it takes a while for the SEs to figure out what is what. Patience is a virtue - and required.
I'm also not sure what plugin you are using but in my mind, it doesn't make sense to use a plugin. It seems to me that if it's not a mobile aware template, that it must actually be reformatting your content in one way or another. Your spiders get leery about spotting something that will deliver one thing to one viewer and something else to another. A responsive WP theme doesn't make content changes at all, it just changes the way it is displayed on the screen, so it's not in danger. If it were me, I would lose the plugin and just use a responsive theme. The last two WP default themes (2014 and 2013, I think?) are responsive. My personal favorite is Mantra (search for it in the theme finder and it'll show up). It's got a lot of whistles and bells without overriding any of your normal functionality and it does a nice job at automatically handling most plugins and getting them to properly scale and swap, too. (The only plugins that sometimes get funky are image display ones that use their own CSS instead of the main WP css - if you run into that trouble when doing it - post on here and I'll be happy to give you a few quick tips on fixing that - I know Kim (cre8pc) is looking for responsive design tips/tricks too - so we can help a bunch of people all at the same time).
Hope that helps.