"A weak soap solution is ideal for dispatching little buggies like the ones which are eating your plant.
Make sure to use real soap and not detergent (most commercial dish soaps are actually detergents). For example, Dr. Bronner's Castile Soap is real soap and is certified organic... perhaps you already have some around the house.
Mix a tablespoon or two in a quart of water and give the plant a good spray. The soap will dehydrate (and therefore kill) the bugs and, in small quantities, will not harm the plant -- or render your peppers poisonous.
Other natural pest repellents include onions, horseradish, cayenne, and garlic, which is particularly good for repelling deer"
Good Question: Bye, Bye Greenfly? [via Apartment Therapy]