[PATCH v6 01/14] kexec: Fix CMA segment address translation with non-zero text_offset

[Jinjie Ruan]

kimage_load_cma_segment() and kimage_map_segment() both translate
a CMA segment using page_address(cma), which returns the CMA base
address. This ignores segment->mem.

On arm64, image_load() adds text_offset to segment->mem before the
segment is loaded:

	kernel_segment->mem += text_offset;
	kernel_segment->memsz -= text_offset;
	image->start = kernel_segment->mem;

so segment->mem no longer matches the CMA base.  The kernel payload is
therefore copied to the wrong offset, while image->start points past
it, and kexec jumps into the middle of the kernel.

kimage_map_segment() has the same problem for any CMA segment whose
mem has been moved.

Add kimage_cma_vaddr() to translate a boot physical address inside a
CMA segment to its virtual address, and use it in both places.

Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
Cc: Baoquan He <baoquan.he@xxxxxxxxx>
Cc: Mike Rapoport <rppt@xxxxxxxxxx>
Cc: Pasha Tatashin <pasha.tatashin@xxxxxxxxxx>
Cc: Pratyush Yadav <pratyush@xxxxxxxxxx>
Cc: Pingfan Liu <piliu@xxxxxxxxxx>
Cc: Justinien Bouron <jbouron@xxxxxxxxxx>
Cc: Sourabh Jain <sourabhjain@xxxxxxxxxxxxx>
Cc: Breno Leitao <leitao@xxxxxxxxxx>
Cc: stable@xxxxxxxxxxxxxxx
Fixes: 07d24902977e ("kexec: enable CMA based contiguous allocation")
Signed-off-by: Jinjie Ruan <ruanjinjie@xxxxxxxxxx>
---
 kernel/kexec_core.c | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c
index dc770b9a6d05..7f7cb77f0caa 100644
--- a/kernel/kexec_core.c
+++ b/kernel/kexec_core.c
@@ -738,11 +738,23 @@ static struct page *kimage_alloc_page(struct kimage *image,
 	return page;
 }
 
+/*
+ * Translate a boot physical address inside a CMA segment to a kernel
+ * virtual address.  Architecture loaders may move segment->mem away from
+ * the CMA base (arm64 adds text_offset), so the offset must be preserved.
+ */
+static void *kimage_cma_vaddr(struct page *cma, unsigned long mem)
+{
+	unsigned long cma_base = page_to_boot_pfn(cma) << PAGE_SHIFT;
+
+	return page_address(cma) + (mem - cma_base);
+}
+
 static int kimage_load_cma_segment(struct kimage *image, int idx)
 {
 	struct kexec_segment *segment = &image->segment[idx];
 	struct page *cma = image->segment_cma[idx];
-	char *ptr = page_address(cma);
+	char *ptr = kimage_cma_vaddr(cma, segment->mem);
 	size_t ubytes, mbytes;
 	int result = 0;
 	unsigned char __user *buf = NULL;
@@ -965,7 +977,7 @@ void *kimage_map_segment(struct kimage *image, int idx)
 
 	cma = image->segment_cma[idx];
 	if (cma)
-		return page_address(cma);
+		return kimage_cma_vaddr(cma, image->segment[idx].mem);
 
 	addr = image->segment[idx].mem;
 	size = image->segment[idx].memsz;
-- 
2.34.1