The starting point here is that history is inevitable because it happened. One can indulge in what-if fantasies but they have no practical relevance. So, British empire is a fact one can't deny, and there is no point mulling over whether India, the country I come from, would have been better or worse off without the empire.
At the same time, it is worth mentioning that history is usually a complex and discreet process. While 'creating history' is a popular expression, it is usually not created consciously or in a moment, though it may appear to have happened in that way with the benefit of hindsight. So, while talking about empire now may make it seem like a grand design or a great conspiracy, it was usually not so during the time it happened. The course of empire, now captured as a neat sequence in history books, was always far more chaotic, a sequence of unrelated actions and decisions made by people without regard to 'legacy'.
Let me stop and illustrate this point for a moment. Let's talk about Churchill, the last great hero of the empire. Looking at him today, we know that he was deeply conscious of his legacy. However, it was legacy in his own terms. By today's standard, he was a bigoted anti-Semitic racist bully, but none of these value judgements existed at the time of his actions, and definitely not at the time of his formative experiences. Most of his political life, he was seen as a mediocre politician with erratic views, destined to have a backbench career in a democracy increasingly dominated by the middle classes: But history presented an opportunity and Britain needed all his chivalrous energy to withstand the Nazi onslaught, giving him his moment of glory and his legacy.
So, there is little logic in the greatness of the British nation and its high moral purpose: It is as phony as the corporate mission statements invented post facto. One lesson in human history is that actors, almost always, act expediently rather than morally, and morality is usually an argument invented later. So, the trail of nonsense from white men's burden to fight for freedom may have its purpose, but has nothing to do with the high moral ground it claims.
For the nations which Britain ruled, the value judgements are more difficult to make and the question of whether these would have been worse off without the imperial intervention invariably crops up. But, in a way, that is a pointless question: As experiences post-imperialism has shown, the rule of domestic elite can be as oppressive as the imperial masters. Framed in context, the question is whether a society is better off with freedom or without it, whether one should give up freedom for progress, at least for a little while. In this regard, the debate is not settled yet: conservative argument in the post-war age suddenly discovered the love of freedom over progress.
But, in a way, it should never be so. As post-imperialism experiences of countries have shown, the lasting legacy of imperialism is to steal the ability of critical thinking from the societies by endowing it with a lasting legacy of division and technocracy. People like me is possibly materially better off by learning English and computers and by being able to participate in the affairs of world commerce; but at the same time, this comes at a cost of disenfranchisement and exclusion of many citizens from my home country's political process, which is invariably dominated by a small political elite created as a direct consequence of imperialism.
So, imperialism may have made the world flatter, but that flatness may be somewhat vertical - a steep slope as seen from most vantage points. Notwithstanding its rhetoric of convenience, it may indeed have promoted the primacy of material progress over 'freedom', as in being one's own people. However, as time goes, the shortcomings of imperial legacy become clearer: It is degenerative not to be able to think critically. The societies touched by imperialism have stalled, and have degenerated into technocratic mediocrity.
Imperial armies and ideologies were powerful, but not powerful enough to deprive us of our humanity.