Ancestry search with the detail you gave gives:
61 England Census
1860s
View Image
Name: Henry James Bailey
Birth: abt 1853 - Marylebone, Middlesex, England
Residence: 1861 - St Pancras, Middlesex, England
1861 England Census
1860s
View Image
Name: James Henry Bailey
Birth: abt 1857 - Reading, Berkshire, England
Residence: 1861 - Reading, Reading St Giles, Berkshire, England
1861 England Census
1860s
View Image
Name: James Henry Bailey
Birth: abt 1859 - Ringwood, Hampshire, England
Residence: 1861 - Ringwood, Hampshire, England
1861 England Census
1860s
View Image
Name: Henry James Bailey
Birth: abt 1860 - Woodborough, Wiltshire, England
Residence: 1861 - Woodborough, Wiltshire, England
1861 England Census
1860s
View Image
Name: James Henry Bailey
Birth: abt 1860 - Sundon, Bedfordshire, England
Residence: 1861 - Sundon, Bedfordshire, England
1861 England Census
1860s
View Image
Name: James Henry Bailey
Birth: abt 1860 - North Bierley, Yorkshire, England
Residence: 1861 - North Bierley, Yorkshire, England
1861 England Census
1860s
View Image
Name: James H Bailey
Birth: abt 1852 - Middlewich, Cheshire, England
Residence: 1861 - Middlewich, Cheshire, England
1861 England Census
1860s
View Image
Name: James H Bailey
Birth: abt 1852 - Scotland
Residence: 1861 - Battersea, Surrey, England
1861 England Census
1860s
View Image
Name: James H Bailey
Birth: abt 1858 - West Ham, Essex, England
Residence: 1861 - Bungay, Bungay St Mary, Suffolk, England
1861 England Census
1860s
View Image
Name: James H Bailey
Birth: abt 1858 - Taunton, Somerset, England
Residence: 1861 - Taunton, St Mary Magdalen, Somerset, England
Which one is him? - none were born in Birmingham - and no birth record for James H Bailey in Bham in 1855 either.