At this stage, when your baby's lips are touched, his tongue automatically moves forward. This is when he will push any food placed on his tongue out of his mouth. This is completely normal.
What you need to know
Your baby is born with a sucking reflex, and tongue thrust is what he will use for feeding from the breast or bottle during his first few months of life. When he pushes the food from his tongue it just means he is not capable of swallowing it. The extrusion of tongue begins to disappear somewhere between four and six months and from this point you can give him some simple food.
What you can do
During this period, you can give him some semi-liquid food like mashed ripe banana, boiled mashed potato, soft idli soaked in water. Care to be taken that only one particular type of food is fed in very little quantity, say 4 to 5 baby spoons for a day and wait to observe that the food accepts his system. Repeat the same type of food for a couple of days and then feed him with another type of food. Semi-liquid foods should not replace breast milk, they should only complement them.