You can modify the field layout by clicking the Modify button under the Field Layout options.
This will allow you to click and drag individual points on the field layout polygon to adjust the size and location of the field layout.
If you would like to delete a field layout, you can click on the Remove button in the same menu seen above. You can also Remove all field layouts within a particular field layer.