In the strictest sense, these are all pretty much the same thing. There are some subtleties as to when you should use one word or the other, but for the purposes of my blog I will probably use them mostly interchangeably, except that I usually try to refer to corporate logos, social network site avatars, and clothing/uniform insignias. Feel free to critique if you think I use the terms too liberally.
In various places of employment I have been called upon to create, or recreate, corporate logos for use in technical documentation and marketing materials. I have also created social network site avatars for use by myself and others. Recently I have created logos for my blog and my original comic strips, and have been experimenting with unofficial prototype designs for other logos.
Logos, emblems, avatars, and insignias generally serve the same purpose, which is to be a simple eye-catching image that is immediately and universally identifiable as to the person, property, corporation, or ideology that it represents. Some logos catch on immediately, and others take time to become universally recognized. A logo can be anything from simple or stylized text to an image or a combination of both.
Over the course of the next series of blog posts I am going to discuss samples of past and present work from my portfolio, offer helpful information on what you should expect when approaching someone for professional logo design, and perhaps persuade you to consider contacting me for your logo design needs.
Be sure to also check out the Professional Logo Design FAQ post for some common questions answered.