Yes. You can choose to publish the form as a Floating Button. Open the Form Editor and go to Publish → Embed → Lightbox.
Afterward, you can customize your button by selecting the text you want to be displayed, the background color, and whether you want the button to be placed on the left side or the right side of your window. Copy the code that was generated and paste it on your website.