Pandas convert string to int

I have a large dataframe with ID numbers:

ID.head()Out[64]: 0    48061050170871    48061050170872    48061050170873    49012950300894    4901295030089

These are all strings at the moment.

I want to convert to int without using loops - for this I use ID.astype(int).

The problem is that some of my lines contain dirty data which cannot be converted to int, for e.g.

ID[154382]Out[58]: 'CN414149'

How can I (without using loops) remove these type of occurrences so that I can use astype with peace of mind?