We've been staying at OKW for years, and really like it for our children who are soon to be 4 and 6. If you've got a one or two-bedroom, the spaciousness is wonderful, and bedrooms separate from the living area allows you to do things while the babies sleep. The kitchen is great and helps cut down on food costs by allowing you to have snacks, breakfast, drinks in the room. If your babies are on formula or baby food, having the kitchen appliances available is great. The washer/dryer come in handy for washing kid clothing that tends to get dirty so quickly.
If you are in a studio, you still retain a lot of advantages. The OKW studio is a good sized room and it's angled layout makes it feel much larger than a regular hotel room. You'll have a microwave, refrigerator, toaster available. The bathroom is large and roomy, a plus when giving kids a bath. The studio also has a small balcony so you and DH can sit outside while the kids sleep. The beds are queen-size and are high off the floor, providing ample room to store your luggage.
As for resort amenities, the main pool area would work well for your children's ages. There is a main pool with a toddler/baby pool beside it. A sandy playground is adjacent to the toddler/baby pool, and my kids have spent many a busy hour running back and forth between the two, playing with friends they have just met at the pool. You can request life preservers for your children and keep them for the length of your stay, which is a boon for swimming in the deeper water of the main pool.
There is no food court at OKW. You have Olivia's, which is a full service sit-down restaurant. Accessible to the boardwalk/turtle krawl area is the bar and Goods to Go, a fast-food service counter. All of these are near the main pool. There is also a general store in which you can buy some provisions, but you can usually do better pricewise if you stop someplace outside of WDW property.
Hope this helps!