Heat 2 tablespoons oil in a large skillet over medium heat. Add squash and cook, covered, stirring occasionally, until tender and lightly browned, 8 to 10 minutes. Add peppers and onion and cook, uncovered, stirring occasionally, until tender, about 5 minutes. Remove from heat and stir in beans, 2 tablespoons cilantro and chile powder. Let cool for 5 minutes.