How does "Allow search engine" work?

So anyone who configures their browser with user-agent Googlebot has access to the articles?

Yes, that's correct. If you don't want to allow Googlebot to access to these articles, better set that option to No

Tuan

I set "Allow search engine" to YES but when I use totheweb.com/learning_center/tools-search-engine-simulator/ to see the page it says it is blocked. Does "Allow search engine" not work anymore?

Search engine can only reach public accessible items. So if access level of the items are Registered (or higher access level), Joomla protects these items itself and search engine could not reach it

Tuan

I dont know what you mean.

My articles are not set to "registered". I restrict access to the articles with MSpro. So why do you put "Allow search engine" in when does not work on restricted articles?

EDIT: I just checked with the Google search console and even the Google bot gets redirected to the subscription page.

Hello

What's access level of these articles? Are they set to Public? If you disable Membership Pro restriction, is this page accessible for public users (users can access to the page without having to login)?

Tuan

What's access level of these articles? Public

When I disable "Plugins: Membership Pro Articles Restriction" all articles are visible without login.

If so, it should works unless there is a problem with Joomla API for detecting search engine

if(preg_match('/http|bot|bingbot|googlebot|robot|spider|slurp|crawler|curl|^$/i', $userAgent))
{
$this->robot = true;
}
else
{
$this->robot = false;
}

I don't know if the tool you are using is reliable, guess we will have to find a way to set custom user-agent from browser for testing

Tuan

I just checked with the Google search console and even the original Google bot gets redirected to the subscription page.

For this, I will have to write some code to test the user-agent string passed by that tool to the page to see whether it's valid

I will update you with the result (hopefully on tomorrow)

Regards,

Tuan