Heat oil in a large pot or dutch oven over medium heat. Saute bacon, onion and potatoes for 10 minutes, stirring frequently.
Add broth and lay fish on top of potatoes in pot. (Don’t worry about cutting it up. It will flake apart when cooked)
Add the Old Bay, salt and pepper. Bring the mixture to a boil, reduce heat to medium, cover and cook until fish and potatoes are cooked through, about 10 minutes. Use a spatula to break the fish into chunks.
Reduce the heat and add the milk or half and half. Heat through but don’t boil.