Uranium doesnt need to undergo fission to be toxic. Fission also occurred naturally in the oklo nuclear reactor long ago. Uranium mined from that area is depleted in U235 and there are higher concentrations of stable isotopes derived from fission products in that area. Arsenic is found in higher concentrations in rice crops. Its found in certain soils and lakes. In certain areas in India, Fluoride can be high enough in concentration to cause bone growth abnormalities. Selenium is found in higher concentrations in the western US to the point that certain plants take it up and concentrate it further up to 2% dry weight. The plants use it as a defense against herbivory. Some trees concentrate nickel to the point that it turns their sap blue and may be a viable source of the element. i.e biomining. The plants that take up selenium also make an alkaloid called swainsonine that if ingested in high enough quantities, can cause cattle and other animals to shake themselves to death. Hence they are colloquially named locoweed i.e crazy weed. Certain plants were historically used as a form of crude birth control due to some of the compounds found in them being abortificants. Echinacea was pulled from the market as it was found to significantly increase the risk of heart attack and stroke due to its stimulant properties. Foxglove was used to develop digitalis which is a valuable heart medication but the plant itself is fairly dangerous. Metformin was derived from naturally occurring compounds that are poisonous in the concentrations they are naturally found in due to their tendency to cause severe hypoglycemia. There are TONS of plants that contain hepatotoxic compounds (cause liver damage). Green potatoes, rhubarb, raw red kidney beans, those all have substances in parts of them that can cause illness.

The point is that nature has plenty of ways to kill. Something being "natural" is no guarantee of safety.