The Mariners won three of four from the Angels over the weekend to earn their first series victory of the year.
Bay tied it in the seventh with his second homer, connecting on the first pitch of the inning from Jason Vargas (0-3).
Morse put Seattle ahead in the eighth against Vargas, hitting his seventh home run into the Angels' bullpen.
Mariners starter Hisashi Iwakuma struck out eight over six innings, allowing only an unearned run.
Carter Capps (2-1) got the victory and Tom Wilhelmsen closed for his ninth save.
Vargas pitched his eighth career complete game in the loss against his former team. He allowed six hits, two walks and struck out seven.
The Angels took a 1-0 lead in the sixth after Andrew Romine led off with a single and moved up on a sacrifice bunt by Peter Bourjos.
Romine scored when a groundball from Mike Trout scooted underneath the glove of shortstop Robert Andino for an error. Andino has been playing more regularly for Seattle since defensive stalwart Brendan Ryan was benched for his struggles at the plate.
Iwakuma left after the sixth inning. He gave up just three hits and walked none.
NOTES: Mariners OF Michael Saunders could rejoin the team on Monday following a Triple-A rehab stint. He sprained his right shoulder on April 11 and was placed on the 15-day disabled list. Saunders played in the field for the time for Triple-A Tacoma on Sunday. ... Iwakuma lowered his ERA to 1.67 for the season.