Adding a function that dynamically generates a robots.txt  to Kohana 3 is actually very easy.

First we need to add a route to bootstrap.php that will route the yourdomain.com/robots.txt url to the function.

Route::set('robots', 'robots.txt')
 ->defaults(array(
  'controller' => 'home',
  'action'     => 'robots',
 ));

Now we create the action “action_robots” in the home controller and output the exclusions we are looking for.

public function action_robots()
{
	
		header('Content-type: text/plain');
		echo "User-agent: *\r\n";
		echo "disallow: /";
		exit();
}

The content-type here is very important, as search engines will not read it as a php file.

NOTE: This will not work if your application if the index_file configuration in bootstrap.php is set to TRUE. This is because if it is set to TRUE then the url would turn out to be /index.php/robots.txt which is obviously not where the search engines will be looking for the restrictions.