Question
We moved into a new house and noticed that in addition to an apple tree we have another tree that produces fruit of some kind. My kids like to eat the (extremely bitter and hard) 'berries' and haven't had any adverse reactions to them. But I'd like to know what the tree is. It isn't very tall, about 2-3 ft taller than an apple tree. It is early september and the 'berries' are green turning reddish orange. i haven't seen any thorns on the tree and it grows naturally in a kind of rounded triangular shape. The branches widen out the lower down the tree you come. The leaves are simple, un-lobed with small teeth along the edge. They are somewhat elliptically shaped coming to a tapered point at the end and rounding at the base.  

Answer
Without a picture I would guess that this is  Flowering Crabapple Trees. Crabapple trees are closely related to apple trees (also Malus) grown for fruit. The main difference is that crabapple fruit is 2 inches diameter or less. Many crabapples develop showy displays of fruit, increasing their ornamental value in the landscape. Some crabapple varieties have more or longer-lasting ornamental value from their fruit display than from their flowers. Fruit varies in size and color, from 1/4 inch to 2 inches and from yellow and orange to purple to brilliant red. Some varieties drop fruit upon ripening in fall; the fruit of many other varieties persists into late winter. Larger-fruited crabapples are valued for making jams and jellies.
They are tart but good to eat. Deer and other animals like them.