Polyurethane glues requires some moisture for curing and a tight fit. Moisten the dowels with water, add a drop or two of glue, and then hammer them into the holes.