This code throws exception:
var query = services
.SomeQuery(bar).select(x => (Foo)x)
.Where(x.PropertyOfFoo == FooState.SomeState);
var result = query.ToList();
The exception:
Unable to cast the type...
LINQ to Entities only supports casting EDM primitive or enumeration types.
This code works:
var query = services
.SomeQuery(bar).select(x => x as Foo)
.Where(x.PropertyOfFoo == FooState.SomeState);
var result = query.ToList();
Why does as allow the conversion and cast does not?
I understand that as will return null and cast would throw an exception if either call fails. Okay. But when I run this code:
var query = services
.SomeQuery(bar);
var result = query.ToList();
I get a much larger query result. Why?
LINQ to Entities is not the same as LINQ to Objects. While LINQ to Objects functions can take any matching delegate and blindly invoke it as normal C# code, LINQ to Entities treats your lambdas as expression trees because it needs to understand the semantics of those lambdas (not just their signature) and transform them to an equivalent query for your EF backend. Naturally, this means that LINQ to Entities can't handle all operations that LINQ to Objects can.
Now, as you have said, one difference between casting and using as is that casting throws an exception on failure and as returns null. But there is an even more important difference between the two: a cast will apply any potential custom explicit conversions, while as will simply attempt to reinterpret the reference as something else, ignoring any potential conversions.
As you understand, a cast is much more complicated than as, because a cast may invoke a custom method (the explicit operator) that's not easily resolvable by the LINQ provider. It's likely that the provider is simply unable to examine the code of the potential custom conversion (I don't know enough about the limitations of expression trees), let alone translate it for the underlying source. Thus, LINQ to Entities chooses to allow only as and the simplest cast cases (basically, it allows the cases where the conversion logic was known ahead of time and can't possibly be custom user code).
There's an important difference between the two statements that shows that more than anything (like CLR rules) Entity Framework is deeply involved here.
x as Foo is translated into SQL. EF can do that because it knows it will always return a result, either a Foo or null. The generated SQL is monstrous by the way, but it does the job.
(Foo)x is not translated into SQL. EF knows that there is no way to create Foo objects for all x's, as cast semantics demand, and it throws an exception.
Note that EF will accept foos.Select(f => (Bar)f), because it's always possible to create a base type from a subtype. The actual cast is done after the SQL result is received, it doesn't affect the SQL itself.
It will also accept bars.OfType<Foo>().Select(x => (Foo)x) (althought this is fairly useless).
So the error message...
LINQ to Entities only supports casting EDM primitive or enumeration types.
...isn't entirely true. EF does accept casts when they're doable. So inside EF's query translator there's just a bunch of logic to decide whether or not it's worth the effort to generate a SQL statement.
It's a difference in how direct cast works vs. how the as operator works. Direct cast isn't available because you aren't using a value type - it only works for primitives.
Now, your lambda is trying to both select and project - you could break that into two operations, so:
var result = services.SomeQuery(bar).select(x => new Foo() {
SomeProperty = x.SomeProperty,
SomeOtherProperty = x.SomeOtherProperty,
... }).ToList()
As to the greater number of results: you're missing a where clause in there.