In 1884, E. Russell Barbour had a Victorian house and carriage house built on the land he purchased from Portland Savings Bank on the corner of Neal and Carroll Streets.  The house was sold in 1919 to Frank E. Milliken who remodeled the house in 1922 and added a 10 x 24 addition.  Though the original appearance is undocumented, the Colonial Revival renovations had resulted in a unified composition.

The building remained in the Milliken family until February of 1988 when Isabel and Alan Smiles bought the house and set about converting the private home into a place of public accomodation.  The Smiles moved to Portland from Greenwich, Connecticut where Isabel owned and operated an antique shop and design business for fourteen years.  After completing structural renovations such as building private baths for each of the guest rooms and overhauling the kitchen, Isabel began the work of designing the inn's guest rooms and public spaces.   The result was a piquant mix of period furnishings and contemporary artwork along with whimsical hand painted walls.  The inn opened in 1989 to rave reviews by guests and design critics alike.

In September of 2007, Isabel sold the inn to Maine native, Kimberly Swan.  Ms. Swan's love of art and inns made the Pomegranate a perfect choice.  Her vow to remain true to Isabel's original vision while still meeting and exceeding guest's expectations, has kept the Pomegranate Inn ranked as one of New England's top Bed & Breakfast.