Each item has a rare chance of appearing in it's structure, so you will most likely have to visit each structure more than once.
Below is a list of all the structures that have a new item, and what the item does:
Underwater Ruins: Aquatic Embryo
The Aquatic Embryo summons a baby dolphin pet to follow you around. Similar to Terraria's pets, it's invincible and you can only have one of them following you at a time.
Igloo: Curing Kit
Curing Kits can be used to cure Zombie Villagers. Just interact with one, and it'll start curing it without the use of golden apples and strength potions. You'll find 2-4 of them at a time.
Dungeon: Dark Amulet
Using the Dark Amulet will use up some of your HP in exchange for summoning a baby zombie minion. It is immune to the sun, and will protect you from other monsters until it dies.
Desert Temple: Dusty Lamp
Similar to the Aquatic Embryo, the Dusty Lamp can be used to summon a small genie pet that will follow you around. Like the Baby Dolphin, you can only have one of them following you at a time.
Village: Encrusted Hammer
Found in weaponsmith and toolsmith houses in villages, the Encrusted Hammer functions like a sword. It has about the same damage and durability as an iron sword, and can be repaired with emeralds. Attacking mobs with it has a small chance to make them drop emeralds.
Woodland Mansion: Fang Gauntlets
The Fang Gauntlets deal 3 damage, and along with punching, you can right-click (or whatever your devices equivalent of right-clicking is) to use up some durability and summon a circle of fangs around yourself.
Nether Fortress: Fortress Armor
Each piece of armor has the same durability and protection as iron armor. Wearing the full set will provide you with infinite fire protection.
Piglin Bastion: Gilded Arrow 
Holding the Gilded Arrow summons a ghast-like creature that will hover behind you. Right clicking with the arrow will make the creature fire a fireball.
