An approach is proposed for robust online behaviour recognition and abnormality detection based on discovering natural grouping of bebaviour patterns through unsupervised learning and a time accumulative reliability measure. A novel behaviour learning model and a run-time accumulative reliability measure are introduced to determine both the natural groupings of possible normal behaviour classes without manual labelling and when sufficient visual evidence has become available for differentiating ambiguities among different behaviour classes observed online. This ensures behaviour recognition at the shortest possible time and robust abnormality detection. We demonstrate that our approach is advantageous over the commonly used Maximum Likelihood (ML) method which can be error prone.