1. The best way is to use some old AT (not ATX) power supply - these PSU was use in older PC since 80-90th up to 2010/2012 and you still can find thousands of them in garages and old stocks.
- It's convenient because it has a simple power switch and has no internal "standby power" "hot/waiting" circuit.
- It has powerful +5V voltage line, that is better than have powerful 12V as it is in newest ATX.
All you need is to connect its black wire to the common GND and RED line (+5V) - to you circuit (dislplays, LEDs, etc) and devices you want to power.
2. The newer ATX PSU (since 2008) has "standby power function" and power supplies are turned on and off by a signal from the motherboard. So you need to add a "standby" power switch connecting one of the black wires to the green wire on the PSU socket. But the internal supply circuit will be always under voltage even if you will turn the switch off.
So, if you can not find the AT PSU (why?), you can use ATX, just connect the 5V as described above for AT PSU.
3. No need to have any fuses - every PC power supply has internal short circuit protection.
You are not protecting your circuit with a fuse. It can be needed only if you are not sure that the power voltage will not be changing from 5v to 10-20-30.
But, first, you can be sure that will never happen with a PC PSU.
And second - if you will use some cheap garbage power supply that provides unstable, "jumping" voltage, the fuse will also not help. So, forget about it and just use a good power supply (with stable voltage).
4. You will never damage your low-power (low-current) electronics equipment with power supply of any amperage capability.
The only thing that matters is correct and stable voltage. So, if your circuit/board/display consumes a few milliamperes at 5v, but you have PS with 100 Amperes capability and 5v output - it's fine. Every device will "take" from the power line only as much current as it needs.
So, the over-voltage can damage your circuit, not the amperage that you can see on the PSU label - it shows just how many amperes the PSU itself can sustain! (the more the amperage, the more devices can be connected to it).
5. "I found an old working power supply from 350w 13A 5v"
You can use the weakest PC power supply you can find (but it's doubt it will be less than 200 watt) - just see at its 5V amperage capability.
You can estimate how many amperes you need summary: for example, 300 LEDs may consume about 5-10 Ams if they all lit at once.
The 7-segment display and LCD current can be estimated as:
Max7219 / Tm1637 - 5 digits:
7-10 mA - min brightness
70-100 mA - max brightness
data - 1.5-2 mA
backlit - 5-50 mA