Open the account menu on the upper right and select Access and Authorizations. Once you are there, select the physician whom you would like to end their access. This will open up their sharing details. You can turn off all access or select turn off access for specific sections. If you have access turned on, allergies are automatically shared. If you do not even want your allergies to be shared, then turn access off on the master switch above.
Their new status will be effective immediately and reflected in the Access and Authorization list.