The final speaker in this session at the SEASON 2025 conference is Fabian Haak, whose interest is in biases in query suggestions. Search engines are key information gateways, and their search query suggestions guide user searches and can that way also influence users’ opinion formation processes. If query suggestions contain biases, then, this could reinforce societal inequities and perhaps even impact on the broader political landscape.
The present project worked with the names of some 352 US politicians, as well as their various personal, professional, and political attributes; it queried Google and Bing for these names, as well as adding letters a to z to these searches and then collecting the further query suggestions. This produced some 47,000 unique suggestions.
The project then used LLMs to categorise and label these suggestions for their query intent, topic, political stance, ideological stance, and the possible presence of bias. The latter in particular is problematic, since biases are not necessarily explicitly expressed in such queries but rather more implicit and contextual. This is why the project did not seek to directly quantify the level of bias in any one query, but instead engaged in an iterative, LLM-based, pairwise comparison of the level of bias in any two queries, eventually enabling a ranking of queries based on their level of bias. Results of this process were slightly affected by the specific LLM versions used, but the overall patterns remained the same across models.
This, finally, enables an analysis of these query suggestions. Several dimensions are important here: for instance, whether queries appear immediately as a first suggestion connected to the root politician name, or further down the suggestion tree; or how transient or persistent query suggestions are.
Google provided somewhat more biased query suggestions, but the average bias score of biased suggestions was larger in Bing; in both search engines biased suggestions appeared further down the order, kicking in somewhat earlier and harder on Google. On Google, the fame and popularity of a politician also appeared to attract more biased query suggestions; overall, Republicans and younger politicians also attracted more biased query suggestions, and ethnicity also appeared to have a range of diverging impacts on the two search engines.
These data continue to be collected now, on an ongoing basis, to examine further dynamics over time, but this also requires further optimisation of analytical processes due to the volume of data this produces.
