I want to provide a captive portal solution for 25 users, however, I am not sure which license level I need.

Level 4 is limited to 200 HotSpot active users and 20 User manager active sessions.
As I want to use the internal radius server and have 25 users, does this mean with L4 only 20 of them are able to log in concurrently and thus I should need to buy a L5 license?

yes, because with L4 license, when 20 people have logged in, the 21st will not be able to log in