Artemas Ward took ownership of the saltbox house and moved in with his family. Prior to this point they had been living in a different house in the center of town.
The house has not always retained its proud yellow color. During war time, only battleship gray paint was available. The house was painted as part of its ordinary maintenance, and for a time it was forgotten that the house had once been yellow.