A novel graph-based optimization framework for salient object detection is proposed in the paper. Multiple graphs are employed in our optimization framework to better describe a natural scene image. Visual rarity is modeled as a regularization term in our framework to better detect saliency. Experimental results on four datasets with fifteen methods prove the effectiveness of our method.