Dynamic Robots.txt with Web Api 2

Hi everyone,

For a project I’m currently working on I needed a dynamic robots.txt. Because our test environment is public facing we want to keep it from being indexed by Google etc. It took a bit of Googling to find a solution that worked, but in the end it was actually pretty simple.

Here’s the action in one of the API Controllers:

    public class UtilitiesController : CustomBaseApiController
    {
        [Route("Robots.txt")]
        [HttpGet]
        public HttpResponseMessage GetRobotsFile()
        {
            var resp = new HttpResponseMessage(HttpStatusCode.OK);
            var stringBuilder = new StringBuilder();

            if (Helpers.IsProduction())
            {
                // Allow bots in production
                stringBuilder.AppendLine("user-agent: *");
                stringBuilder.AppendLine("disallow: ");
            }
            else
            {
                // Don't allow bots in non-production environments
                stringBuilder.AppendLine("user-agent: *");
                stringBuilder.AppendLine("disallow: *");
            }

            resp.Content = new StringContent(stringBuilder.ToString());

            return resp;
        }
    }

Also need to add the following to your web.config so that the robots.txt file can processed by the routing handler. Without this IIS will attempt to serve it as a static file and will return a 404 when it’s not found:


    
    
        
        
        
    
    

In production you’ll end up with the following:

user-agent: *
disallow:

And any other environments:

user-agent: *
disallow: *

Thanks to these answers on stackoverflow for the info:
https://stackoverflow.com/a/52270877/522859
https://stackoverflow.com/a/17037935/522859

AWS EC2 Elastic Beanstalk Going to Sleep – .Net/Windows/MSSQL Server Express

Hi everyone,

I’ve been having a bit of an issue with my AWS app going to sleep and taking a long time to handle initial requests.

I’m using .NET with Elastic Beanstalk on a T2 Micro Instance and MSSQL Server Express on RDS. My FrontEnd is a static ReactJS app that sits in S3 behind CloudFront. There’s also a load balancer across the backend.

My frontend was always instant but my initial Api calls were timing out. This ruled out S3 and CloudFront, leaving the following:

  • Load balancer
  • RDS/MSSQL
  • EC2/IIS

After a bit of Googling I came across something that looked fairly promising – MSSQL Server Express has a property called AutoClose set to ‘ON’ by default. AWS appears to correct this as mine was off however it’s worth checking:

-- If set to zero then auto close is off
SELECT DATABASEPROPERTY('mydatabasename','IsAutoShrink')

-- Check all instances at once
SELECT name,is_auto_close_on FROM sys.databases

-- Turn off if on
ALTER DATABASE myDB SET AUTO_CLOSE OFF

In my case the actual problem turned out to be IIS idle timeout. By default IIS automatically times out an application after 20 minutes. In order to disable this create the following ebextension:

commands:
    setIdleTimeoutToZero:
        cwd: "C:\windows\system32\inetsrv"
        command: "appcmd set apppool /apppool.name:DefaultAppPool /.processModel.idleTimeout:0.00:00:00"

If you haven’t done this before, all you need to do is create a folder called .ebextensions under your project directory. Then create a new file called iis-idle-timeout.config and add the yaml to it. If you need more information on iis timing out check out this great blog post: https://notebookheavy.com/2017/06/21/set-iis-idle-timeout-elastic-beanstalk/

Thanks to these sources for the solutions:
Disable auto_close: https://stackoverflow.com/a/1750400/522859
IIS Timeout: https://notebookheavy.com/2017/06/21/set-iis-idle-timeout-elastic-beanstalk/
EBExtension Info: http://notebookheavy.com/2017/05/01/auto-install-newrelic-agent-elastic-beanstalk/

Web API 2 – ExceptionMessage=No MediaTypeFormatter is available to read an object of type ‘HttpPostedFileBase’ from content with media type ‘multipart/form-data’.

Hi everyone,

I ran into the following error while trying to get image uploads working with Web API 2:

ExceptionMessage=No MediaTypeFormatter is available to read an object of type ‘HttpPostedFileBase’ from content with media type ‘multipart/form-data’.

I had been trying to copy the following from an mvc controller in another project:

public IHttpActionResult Upload(HttpPostedFileBase file, Models.Image.ImageAssociationType associationType, int associationId)

The fix was to use the following instead:

public IHttpActionResult Upload(Models.Image.ImageAssociationType associationType, int associationId)
{
var file = HttpContext.Current.Request.Files.Count > 0 ? HttpContext.Current.Request.Files[0] : null;

Thanks to this stackoverflow post for the info: https://stackoverflow.com/a/28370156/522859

DbSet does not contain a definition for ‘FromSQL’ and no extension method ‘FromSql’ accepting an argument of type ‘DbSet’ could be found.

Hi everyone,

I ran into the following error while attempting to use a custom query with EntityFramework:

DbSet does not contain a definition for ‘FromSQL’ and no extension method ‘FromSql’ accepting an argument of type ‘DbSet’ could be found.

This one’s pretty straight forward:

// Install the following package via nuget
Install-package Microsoft.EntityFrameworkCore.Relational

//Add the following namespace to your file
using Microsoft.EntityFrameworkCore;

Hopefully that’ll solve it for you, but if not there’s a lot a more information in these posts:
https://stackoverflow.com/a/38919326/522859
https://github.com/npgsql/Npgsql.EntityFrameworkCore.PostgreSQL/issues/146

Include UserId in Login Response (Token) – Web API 2

Hi everyone,

A quick post on how to include the user’s id in your login response when using Web API 2.

The default response to the /Token request is as follows:

{
“access_token”: “xxxxxxxxxxxxx_xxxx”,
“token_type”: “bearer”,
“expires_in”: 1209599,
“userName”: “test@test.com”,
“.issued”: “Mon, 23 Apr 2018 06:08:03 GMT”,
“.expires”: “Mon, 07 May 2018 06:08:03 GMT”
}

Once the changes below have been made the response will include a userId field:

{
“access_token”: “xxxxxxxxxxxxx_xxxx”,
“token_type”: “bearer”,
“expires_in”: 1209599,
“userName”: “test@test.com”,
“.issued”: “Mon, 23 Apr 2018 06:08:03 GMT”,
“.expires”: “Mon, 07 May 2018 06:08:03 GMT”,
“userId”: “xxxxxxx”
}

There are three very small changes required in order to add this functionality.

First, add an additional argument to CreateProperties in ApplicationOAuthProvider.cs

public static AuthenticationProperties CreateProperties(string userName, string userId)
{
IDictionary data = new Dictionary
{
{ "userName", userName },
// Add
{ "userId", userId }
};
return new AuthenticationProperties(data);
}

Pass userId to CreateProperties in ApplicationOauthProvider.

// ApplicationOAuthProvider.cs > GrantResourceOwnerCredentials
ClaimsIdentity oAuthIdentity = await user.GenerateUserIdentityAsync(userManager, OAuthDefaults.AuthenticationType);
ClaimsIdentity cookiesIdentity = await user.GenerateUserIdentityAsync(userManager,
CookieAuthenticationDefaults.AuthenticationType);

// Add user id
AuthenticationProperties properties = CreateProperties(user.UserName, user.Id);
AuthenticationTicket ticket = new AuthenticationTicket(oAuthIdentity, properties);
context.Validated(ticket);

Pass userId to CreateProperties in AccountController.

// AccountController.cs > GetExternalLogin
if (hasRegistered)
{
Authentication.SignOut(DefaultAuthenticationTypes.ExternalCookie);

ClaimsIdentity oAuthIdentity = await user.GenerateUserIdentityAsync(UserManager, OAuthDefaults.AuthenticationType);
ClaimsIdentity cookieIdentity = await user.GenerateUserIdentityAsync(UserManager,
CookieAuthenticationDefaults.AuthenticationType);

// Add userid
AuthenticationProperties properties = ApplicationOAuthProvider.CreateProperties(user.UserName, user.Id);
Authentication.SignIn(properties, oAuthIdentity, cookieIdentity);

Cannot attach the file ‘C:…database.mdf’ as database x – Entity Framework

Hi everyone,

I ran into the following error when attempting to run ‘update-database’ on an initial migration:

Cannot attach the file ‘C:Users…App_Dataaspnet-…115933.mdf’ as database ‘aspnet-…15933’

The solution to this one is pretty easy, remove the initial catalog property from your connection string.

…33.mdf;Initial Catalog=aspnet…

This is apparently caused by issues with EntityFramework and multiple projects in the same database. See the following stackoverflow answer for more info: https://stackoverflow.com/a/20176660/522859

Retrieving User Id in Web API 2 Controller – .NET

Hi everyone,

Just a quick post on how to retrieve the current user’s id in a Web API 2 controller:

var userId = RequestContext.Principal.Identity.GetUserId();

Note that you’ll need the following using statements:

using Microsoft.AspNet.Identity;
using System.Web.Http;

Thanks to the following stackoverflow post for the info: https://stackoverflow.com/a/21618056/522859