If there is a way to segment all the contact you need to update a field, and that is the same value, then you can do that by an automation - you can bulk add a tag that triggers automation - and that automation update the field.
But if it is not so simple, different field values, or hard to segment them in AC - the other way is add contacts with field values from file - but NOT add them to any list, and add specific tag - then - after import - you can easily find the contacts with this tag and not on any list - and just delete them (export and paste into list cleanup- bulk removal) - a bit crazy - but works if it is more than a few contacts 