Skip to content

Conversation

@iamcodingcat
Copy link
Contributor

What this PR does / why we need it:

I run go-based feature serving server and my server is listen on both HTTP and gRPC. I mainly use gRPC server on production environment. But if some entity join key not exists in expected join keys from feast registry, Calling function that fetch online features from online-store(GetOnlineFeatures), the gRPC server is on panic . The panic is critical for production. If someone know this issue, a attack like DDoS is possible.

For example, suppose someone who knows about this vulnerability keeps sending requests to my Feast gRPC feature serving server, specifying an entity join key that does not exist in my Feast registry. Then, every time such a request occurs, the gRPC feature serving server will panic, and eventually, it won't be able to process any of the normal requests that the server is supposed to handle. In fact, as long as the entity join key simply doesn't exist in the Feast registry, the attack would succeed even if they blatantly specify a key like "my_name_is_hacker".

So, I add validation logic that check if input entity join key exists in expected entity join keys for preventing panic.
For reference, the HTTP server not panic without my added logic(this lines raise error in it). But if the HTTP server includes the validation logic, debug is easier for feast users.

I attached the stack trace when the panic occurs. the panic occurs in this line.

panic: assignment to entry in nil map

goroutine 42 [running]:
github.com/feast-dev/feast/go/internal/feast/onlineserving.ValidateEntityValues(...)
	/Users/zedd/Desktop/private/feast/go/internal/feast/onlineserving/serving.go:280
github.com/feast-dev/feast/go/internal/feast.(*FeatureStore).GetOnlineFeatures(0x14000948930, {0x106985b38, 0x14000522660}, {0x140007a7640, 0x2, 0x2}, 0x0, 0x14000949b00, 0x0, 0x0)
	/Users/zedd/Desktop/private/feast/go/internal/feast/featurestore.go:130 +0x288
github.com/feast-dev/feast/go/internal/feast/server.(*grpcServingServiceServer).GetOnlineFeatures(0x140007b1758, {0x106985b38?, 0x14000948ae0?}, 0x140004a80a0)
	/Users/zedd/Desktop/private/feast/go/internal/feast/server/grpc_server.go:50 +0x1b4
github.com/feast-dev/feast/go/protos/feast/serving._ServingService_GetOnlineFeatures_Handler({0x10674ff20, 0x140007b1758}, {0x106985b38, 0x14000948ae0}, 0x140005a5080, 0x0)
	/Users/zedd/Desktop/private/feast/go/protos/feast/serving/ServingService_grpc.pb.go:135 +0x1c0
google.golang.org/grpc.(*Server).processUnaryRPC(0x140009a2000, {0x106985b38, 0x14000948a50}, 0x140009385a0, 0x14000984240, 0x10786a9f8, 0x0)
	/Users/zedd/go/pkg/mod/google.golang.org/grpc@v1.75.0/server.go:1431 +0xc9c
google.golang.org/grpc.(*Server).handleStream(0x140009a2000, {0x1069864f8, 0x1400092f040}, 0x140009385a0)
	/Users/zedd/go/pkg/mod/google.golang.org/grpc@v1.75.0/server.go:1842 +0x900
google.golang.org/grpc.(*Server).serveStreams.func2.1()
	/Users/zedd/go/pkg/mod/google.golang.org/grpc@v1.75.0/server.go:1061 +0x84
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 41
	/Users/zedd/go/pkg/mod/google.golang.org/grpc@v1.75.0/server.go:1072 +0x138

Process finished with the exit code 2

Which issue(s) this PR fixes:

  • Prevent the panic of go-based feature serving server and a intentional attack
  • Enhance error message in both HTTP and gRPC feature serving server

Misc

Copy link
Collaborator

@shuchu shuchu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@shuchu shuchu added ok-to-test go Pull requests that update Go code labels Dec 5, 2025
Signed-off-by: iamcodingcat <joyh951021@gmail.com>
@ntkathole ntkathole force-pushed the fix/go/serving/grpc/validate-entity-join-key branch from 072059f to 4e14b8b Compare December 6, 2025 17:46
@ntkathole ntkathole merged commit 0b93559 into feast-dev:master Dec 7, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

go Pull requests that update Go code ok-to-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants