You have to make sure that the order of the CSV matches the importing columns in outlook.
I.e. the CSV file has:
first name, last name, email address, phone, etc.
The outlook client expects:
email address, last name, first name, home address etc.
What you need to do is make sure to map the fields from the CSV to their matching fields in outlook.
The example I gave has a matching column names. Outlook will auto match the column names.
The problem starts when the CSV has different column names:
field1, field2, field3, etc.
At this point the outlook import is unable to determine what is what which is likely partly responsible for the error you received.