@d-kay you should research some more on batteries and connecting them in series vs parallel there is a ton of information on the forum and on the internet in general about this. You shouldn’t mess with these batteries if you don’t know what you’re dealing with just buy a prebuilt pack that is made for what you want (you’ll still need to have a basic understanding to make an informed purchase).
The answer to your specific question is you add more batteries in series or more cells in series to increase the voltage (stack a couple of AA batteries together an use a voltmeter to see, can also do them in parallel, use some foil and tape can make simple connections to test things with lower voltage safe® stuff). A battery is really just a collection of cells in series (S) or parallel §, with LiPo chemistry the cells have a fully charged voltage of 4.2V and are typically mostly drained at 3.7V. Min voltage per cell is actually around 3V but there can easily be a half volt sag while the battery is under load (supplying power to the motor). Typically RC enthusiasts who want their batteries to last will only usually drain 80% of the total capacity of the battery to avoid dropping any individual cell in the pack below the minimum safe voltage.
When you put batteries in parallel you don’t increase the voltage (they are just both applying their voltage to the lines their attached to), but you are drawing current through both the batteries so you increase the amount of current you can safely draw if you have more cells or batteries in parallel (you also add the Ah in parallel, in series you are only adding voltage not Ah, can think of Ah as amount of fuel in the tank, measured in terms of how much current you can draw for how long before empty). The RC style LiPos are typically rated with a C rating, the C rating relates max safe amperage drawn from the battery to the capacity of the battery. A 5Ah battery with a 20C rating can have 5*20 = 100A current load on it.
For the most part the safety concerns are if you short things out the battery will deliver that 100A+ through whatever is shorting and quickly turn whatever is in the path into vapor (releasing a lot of light and heat in the process, or worse the battery itself goes up). So if you do start working on the batteries or taking off leads be sure you know what you’re doing, you can also buy premade series connectors or parallel connectors to join the batteries together if you aren’t confident in your knowledge of working on the batteries (it’s somewhat high voltage stuff and you’re probably going to need to solder it which means you’re dealing with melting lead/tin on live leads).
Edited a few times to add some details, just added link as well for more explanation on batteries and various chemistries available.