An operator had just replaced the ignition port cover on a hydrochloric acid plant burner. Normal start-up was continued by closing the drain valve in the product run off line and opening the combustion chamber condensate line. At this point a passing mechanic noted that a three foot hydrogen flame was burning from the vent stack. The plant was shut down. No evidence of an explosion or internal burning was identified. No source of ignition was identified for the flame. A small amount of charring occurred on the UPVC stack.