Since you haven't felt you were able to play recorders and xylophones well when you were in school, I recommend that you find someone who knows how to play bass (a teacher or new bass player friend) and arrange to have a few lessons to get you started on the right track in order to boost your confidence.
You can then enhance or replace your lessons by checking out "how to learn bass" books from the library and watching bass videos or other players. And practice, practice, practice.
I'm thinking that either way...with a teacher or without...you will have to buy a bass and amplifier, unless you can borrow one for a while. Have you checked if teachers in your area also have a bass you can use temporarily?
The advantage of having a teacher at the start, at least, is that a teacher can help you avoid problems and steer you in the right direction.
For example, when I decided "I'm going to play the electric guitar" and went out to buy one knowing very little about them, I learned the hard way at the store while trying out the guitar that YOU DON'T UNPLUG THE ELECTRIC GUITAR FROM THE AMPLIFIER WHEN THE AMPLIFIER IS STILL ON!!!! (When you do this you get horrid, noisy feedback.) The music store employee immediately explained, as soon as I did that, why I shouldn't.
I've never made that error again.
So, having some pointers from someone who knows the instrument *before* you do something wrong is definitely helpful.
I have tried an instrument once and had hoped it would be easy, but it wasn't (the trumpet), but that didn't stop me from trying other instruments (cello, recorder) and doing fine. So, I wouldn't use your past experiences with instruments as any kind of prediction for how well you can learn to play a different instrument.
Out of curiosity, how old were you when you were playing the recorder in school, and what do you think caused your playing not to go well? I ask, because there could be many reasons for your experience being bad that have nothing to do with *you*. For example, often school music teachers don't have enough time to spend on individual students to correct errors. Also, the instrument itself might have been a very cheap one, improperly formed, and that could have made the recorder nearly unplayable or simply sound bad even when played by an experienced player.