The general answer to the question of this kind is that there should be no direct evidence that can establish your intent not to work for the employer, or there was no such job at the time of your Green Card approval. The question often pops up in the naturalization proceedings which require to disclose the employment for past five years. 

If such information on employment history indicates that you did not work for the sponsoring employer, or your worked for another employer immediately the date after the Green Card approval date, it should be a red flag for USCIS naturalization adjudicator to see whether the Green Card should have been granted, and whether the Green Card should be revoked. This does not happen all the time, but some USCIS adjudicators can look into this issue.