Your circle is not drawn correctly, because Modifier.border draws a rectangle border by default, and then you clip it with your Modifier.clip. Instead, if you need to apply shape to the border, you need to pass the shape into Modifier.border, like this:

.border(BorderStroke(2.dp, Color.Gray), shape = CircleShape)

But this won't solve your problem. To draw the shadow correctly like shown in your image, you need to apply a custom Shape to your container.

You can use Modifier.onGloballyPositioned to get position of your cutoffs:

var separatorOffsetY by remember { mutableStateOf<Float?>(null) }
val cornerRadius = 20.dp
Card(
    shape = RoundedCutoutShape(separatorOffsetY, cornerRadius),
    backgroundColor = Color.White,
    modifier = Modifier.padding(10.dp)
) {
    Column {
        Box(modifier = Modifier.height(200.dp))
        Box(
            Modifier
                .padding(horizontal = cornerRadius)
                .height(1.dp)
                .requiredWidth(250.dp)
                // DottedShape is taken from this answer:
                // https://stackoverflow.com/a/68789205/3585796
                .background(Color.Gray, shape = DottedShape(step = 20.dp))
                .onGloballyPositioned {
                    separatorOffsetY = it.boundsInParent().center.y
                }
        )
        Box(modifier = Modifier.height(50.dp))
    }
}

Using this information you can create a shape like following:

class RoundedCutoutShape(
    private val offsetY: Float?,
    private val cornerRadiusDp: Dp,
) : Shape {
    override fun createOutline(
        size: Size,
        layoutDirection: LayoutDirection,
        density: Density,
    ) = Outline.Generic(run path@{
        val cornerRadius = with(density) { cornerRadiusDp.toPx() }
        val rect = Rect(Offset.Zero, size)
        val mainPath = Path().apply {
            addRoundRect(RoundRect(rect, CornerRadius(cornerRadius)))
        }
        if (offsetY == null) return@path mainPath
        val cutoutPath = Path().apply {
            val circleSize = Size(cornerRadius, cornerRadius) * 2f
            val visiblePart = 0.25f
            val leftOval = Rect(
                offset = Offset(
                    x = 0 - circleSize.width * (1 - visiblePart),
                    y = offsetY - circleSize.height / 2
                ),
                size = circleSize
            )
            val rightOval = Rect(
                offset = Offset(
                    x = rect.width - circleSize.width * visiblePart,
                    y = offsetY - circleSize.height / 2
                ),
                size = circleSize
            )
            addOval(leftOval)
            addOval(rightOval)
        }
        return@path Path().apply {
            op(mainPath, cutoutPath, PathOperation.Difference)
        }
    })
}

Result:

Get rid of:

shape = DottedShape(step = 20.dp)