How does "Allow search engine" work?

  • jeronimo
  • Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
More
2 months 2 weeks ago #125802 by jeronimo
How does "Allow search engine" work? was created by jeronimo
So anyone who configures their browser with user-agent Googlebot has access to the articles?

Please Log in or Create an account to join the conversation.

More
2 months 2 weeks ago #125812 by Tuan Pham Ngoc
Replied by Tuan Pham Ngoc on topic How does "Allow search engine" work?
Yes, that's correct. If you don't want to allow Googlebot to access to these articles, better set that option to No

Tuan

Please Log in or Create an account to join the conversation.

  • jeronimo
  • Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
More
1 month 4 weeks ago #126291 by jeronimo
Replied by jeronimo on topic How does "Allow search engine" work?
I set "Allow search engine" to YES but when I use totheweb.com/learning_center/tools-search-engine-simulator/ to see the page it says it is blocked. Does "Allow search engine" not work anymore?

Please Log in or Create an account to join the conversation.

More
1 month 4 weeks ago #126293 by Tuan Pham Ngoc
Replied by Tuan Pham Ngoc on topic How does "Allow search engine" work?
Search engine can only reach public accessible items. So if access level of the items are Registered (or higher access level), Joomla protects these items itself and search engine could not reach it

Tuan

Please Log in or Create an account to join the conversation.

  • jeronimo
  • Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
More
1 month 4 weeks ago - 1 month 4 weeks ago #126294 by jeronimo
Replied by jeronimo on topic How does "Allow search engine" work?
I dont know what you mean.

My articles are not set to "registered". I restrict access to the articles with MSpro. So why do you put "Allow search engine" in when does not work on restricted articles?

EDIT: I just checked with the Google search console and even the Google bot gets redirected to the subscription page.
Last edit: 1 month 4 weeks ago by jeronimo.

Please Log in or Create an account to join the conversation.

More
1 month 4 weeks ago #126295 by Tuan Pham Ngoc
Replied by Tuan Pham Ngoc on topic How does "Allow search engine" work?
Hello

What's access level of these articles? Are they set to Public? If you disable Membership Pro restriction, is this page accessible for public users (users can access to the page without having to login)?

Tuan

Please Log in or Create an account to join the conversation.

  • jeronimo
  • Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
More
1 month 4 weeks ago #126300 by jeronimo
Replied by jeronimo on topic How does "Allow search engine" work?
What's access level of these articles? Public

When I disable "Plugins: Membership Pro Articles Restriction" all articles are visible without login.

Please Log in or Create an account to join the conversation.

More
1 month 4 weeks ago #126301 by Tuan Pham Ngoc
Replied by Tuan Pham Ngoc on topic How does "Allow search engine" work?
If so, it should works unless there is a problem with Joomla API for detecting search engine

if(preg_match('/http|bot|bingbot|googlebot|robot|spider|slurp|crawler|curl|^$/i', $userAgent))
{
$this->robot = true;
}
else
{
$this->robot = false;
}

I don't know if the tool you are using is reliable, guess we will have to find a way to set custom user-agent from browser for testing

Tuan

Please Log in or Create an account to join the conversation.

  • jeronimo
  • Topic Author
  • Offline
  • Expert Boarder
  • Expert Boarder
More
1 month 4 weeks ago #126302 by jeronimo
Replied by jeronimo on topic How does "Allow search engine" work?
I just checked with the Google search console and even the original Google bot gets redirected to the subscription page.

Please Log in or Create an account to join the conversation.

More
1 month 4 weeks ago #126303 by Tuan Pham Ngoc
Replied by Tuan Pham Ngoc on topic How does "Allow search engine" work?
For this, I will have to write some code to test the user-agent string passed by that tool to the page to see whether it's valid

I will update you with the result (hopefully on tomorrow)

Regards,

Tuan
The following user(s) said Thank You: jeronimo

Please Log in or Create an account to join the conversation.