Yep hold both, bullets will fly out of your chest cannon, you can even adjust the direction of the bullets with the directional buttons while firing. Do not underestimate it as it is quite powerful when you upgrade it to 3+.
Oh ok that makes sense. I was just tapping them.