I have these two pipelines in my Jupyter notebook.
## Loading Diffusion Pipelinefrom diffusers import DiffusionPipelineimport torch##Image Generatingpipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float32, use_safetensors=True, variant="fp16")pipe.unet = torch.compile(pipe.unet, mode="reduce-overhead", fullgraph=True)# load both base & refinerbase = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float32, variant="fp16", use_safetensors=True)pipe.enable_model_cpu_offload refiner = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-refiner-1.0", text_encoder_2=base.text_encoder_2, vae=base.vae, torch_dtype=torch.float32, use_safetensors=True, variant="fp16",)# Define how many steps and what % of steps to be run on each experts (80/20) heren_steps = 40high_noise_frac = 0.8prompt = "A man playing soccer in a stadium"# run both expertsimage = base( prompt=prompt, num_inference_steps=n_steps, denoising_end=high_noise_frac, output_type="latent",).imagesimage = refiner( prompt=prompt, num_inference_steps=n_steps, denoising_start=high_noise_frac, image=image,).images[0].show()##Image Captioningfrom transformers import pipelineimage_to_text = pipeline("image-to-text", model="nlpconnect/vit-gpt2-image-captioning")image_to_text(image)The first one is creating an image with stable diffusion xl-base and with the other one I want to get an image captioning of the created image.How can I assign the generated image which is saved in the RAM to the image captioning pipeline?
Thanks!
I tried to put the "image" variable into the second pipeline put always received an error.