In August 1963, Brown joined then
non-league Wigan Athletic as player-manager, replacing
Johnny Ball. He remained at Wigan until July 1966. In November 1966, he returned to Luton, this time as manager of a side struggling in the
Fourth Division. He soon changed the fortunes of the Hatters, leading them to the Fourth Division title in
1968. They started the next season promisingly (and were eventually to finish in third place); however, Brown was sacked in December 1968 after it was discovered he had applied for the vacant managerial post at
Leicester City. Brown's departure from Luton is memorialized at the 47:51 mark of "On Her Majesty's Secret Service", when the character Campbell can be seen reading a newspaper with the prominent headline "Why Brown had to go". He was not out of work for long, joining
Torquay United as manager in January
1969. He kept the Gulls comfortably in
Division Three during his first two-and-a-half seasons in charge (finishes of sixth, thirteenth and tenth), but after a poor start to the
1971–72 season, he was sacked in October 1971. Torquay would eventually be relegated later that season. On 20 June 1972, Brown became manager of
Bury, leading them to twelfth place in Division Four at the end of his first season in charge. The following season, 1973–74, Bury started brightly, and would eventually be promoted in fourth place; Brown, however, had left on 19 November 1973, to manage
Nottingham Forest, and took them to seventh place by the end of the season. The following
season, Forest started badly and Brown upset the fans by selling
Duncan McKenzie to
Brian Clough's
Leeds United. He was sacked on 3 January 1975, shortly after a 2–0 home defeat to local rivals
Notts County. Clough replaced Brown at Forest. On 5 May 1976, Brown was appointed manager of another of his former playing clubs,
Blackpool, who at the time were in the
Second Division. The following
season saw them battle for promotion and eventually finish in fifth place, missing out on a return to Division One by two points. During the next season the club were again challenging for promotion, but after an argument with chairman Billy Cartmell, on 6 February 1978 Brown was sacked. Cartmell had made remarks in a daily newspaper about Brown's job being on the line despite five-goal home wins in recent games, to which Brown responded by calling the chairman a "back-stabbing rat", which the board felt was unacceptable. After his departure, Blackpool won only one more game, dropped down the table, and were narrowly relegated at the end of the season. Later that year he moved to
Kuwait to manage Quadsia, but returned to manage
Blackpool in March 1981 as a replacement for
Alan Ball; however, he was unable to prevent their relegation to Division Four for the first time in their history. The following season saw a mid-table finish, and Brown left Blackpool in May 1982. ==Death==